Skip to content

Deploying Open VSX

Jump to bottom
amvanbaren edited this page May 3, 2023 · 10 revisions

Architectural Overview

Overview diagram

The central element is the server application, which is available in the openvsx-server Docker image. It is a Spring Boot application and needs an application.yml file to configure the deployment. Open VSX does not provide any facility to deploy the other components (database, search engine etc.) because there are numerous ways how the infrastructure can be set up. Using Kubernetes is one option, but that is not mandatory.

The database holding all metadata of published extensions is a PostgreSQL instance. In case no additional file storage is used, all files are stored as binary data in the database. Though this setup is supported by Open VSX, it considerably increases storage and network thoughput of the database, so using an external file storage is recommended. Currently Azure Blob Storage and Google Cloud Storage are supported as external storage providers.

Elasticsearch is used as default search engine for search queries from the web UI. As an alternative, you can choose to search via database queries, which will likely result in worse performance, or to completely disable the search functionality. In the latter case, all other Open VSX features are still available.

User authentication is done with OAuth. Currently only GitHub is supported as OAuth provider.

Getting Started

You can quickly spin up an Open VSX server and webui using the DockerFile below. Additionally you need a PostgreSQL instance and an Elasticsearch instance. To use the user and admin sections of the webui, you need to configure a GitHub OAuth app. For troubleshooting and further configuration, see issue #703.

DockerFile

ARG OPENVSX_VERSION

# Builder image to compile the website
FROM ubuntu as builder

WORKDIR /workdir

RUN apt-get update \
  && apt-get install --no-install-recommends -y \
    bash \
    ca-certificates \
    git \
    curl \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

# See https://github.com/nodesource/distributions/blob/main/README.md#debinstall
RUN curl -sSL https://deb.nodesource.com/setup_14.x | bash - \
  && apt-get install -y nodejs

RUN npm install --global yarn@1.*

ARG OPENVSX_VERSION
ENV VERSION=$OPENVSX_VERSION

RUN git clone --branch ${VERSION} --depth 1 https://github.com/eclipse/openvsx.git /workdir
RUN /usr/bin/yarn --cwd webui \
  && /usr/bin/yarn --cwd webui build \
  && /usr/bin/yarn --cwd webui build:default

# Main image derived from openvsx-server
FROM ghcr.io/eclipse/openvsx-server:${OPENVSX_VERSION}

COPY --from=builder --chown=openvsx:openvsx /workdir/webui/static/ BOOT-INF/classes/static/

Building Open VSX Docker image

The command gets the latest release tag name and uses it to build an openvsx image from the DockerFile in the current working directory.

export OPENVSX_VERSION=`curl -sSL https://api.github.com/repos/eclipse/openvsx/releases/latest | jq -r ".tag_name"`

docker build -t "openvsx:$OPENVSX_VERSION" --build-arg "OPENVSX_VERSION=$OPENVSX_VERSION" .

Configuring application.yml

The Open VSX server is configured using an application.yml file. See the Spring Boot documentation for more information on this configuration format.

The server application will automatically load the configuration file if you put it into the directory /home/openvsx/server/config of the server image. There are several ways to do this, e.g. you can extend the Docker image or use a Kubernetes ConfigMap.

You can use all configuration properties offered by Spring to set up your deployment. In particular, you need to configure a datasource to connect the application with the database.

Open VSX Configuration Properties

This section describes the special configuration properties supported by the Open VSX server.

Publishing

Property ovsx.publishing.require-license
Type boolean
Default false
Compatibility Since 0.1

Whether published extensions are required to have a license. If active, unlicensed extensions are rejected.

Web UI

Property ovsx.webui.url
Type string
Default
Compatibility Since 0.1

Base URL of the web UI. This is required only if it's different from the server.

Property ovsx.webui.frontendRoutes
Type string[]
Default /extension/**,/user-settings/**,/admin-dashboard/**
Compatibility Since 0.1

Routes to be forwarded to / because they are handled by the frontend.

Search Options

Property ovsx.search.relevance.rating
Type double
Default 1.0
Compatibility Since 0.2

Weight of user ratings for computing relevance. This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.search.relevance.downloads
Type double
Default 1.0
Compatibility Since 0.2

Weight of download counts for computing relevance. This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.search.relevance.timestamp
Type double
Default 1.0
Compatibility Since 0.2

Weight of publishing timestamps for computing relevance (newer extensions are ranked higher). This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.search.relevance.unverified
Type double
Default 0.5
Compatibility Since 0.2

Relevance factor for unverified extension versions. The combined relevance from the averageRating, downloadCount and timestamp criteria is multiplied with this value if the publisher of the extension is not a member of the extension's namespace or the namespace has no owner.

Elasticsearch

Property ovsx.elasticsearch.enabled
Type boolean
Default true
Compatibility Since 0.1

Whether to enable search functionality though Elasticsearch. By switching this off, it is not necessary to deploy Elasticsearch. Cannot be used together with ovsx.databasesearch.enabled.

Property ovsx.elasticsearch.clear-on-start
Type boolean
Default false
Compatibility Since 0.1

Whether to clear and rebuild the search index on startup. If disabled, the index is built only if it does not exist yet. Rebuilding the search index may take several minutes if there are many extensions.

Property ovsx.elasticsearch.host
Type string
Default localhost:9200
Compatibility Since 0.1

Host and port of the Elasticsearch instance.

Property ovsx.elasticsearch.ssl
Type boolean
Default false
Compatibility Since 0.1

Whether to connect with SSL.

Property ovsx.elasticsearch.username
Type string
Default
Compatibility Since 0.1

Username for basic authentication.

Property ovsx.elasticsearch.password
Type string
Default
Compatibility Since 0.1

Password for basic authentication.

Property ovsx.elasticsearch.truststore
Type string
Default
Compatibility Since 0.1

Path to a trust store file for SSL connection.

Property ovsx.elasticsearch.truststoreProtocol
Type string
Default TLSv1.2
Compatibility Since 0.1

Protocol for SSL connection.

Property ovsx.elasticsearch.truststorePassword
Type string
Default
Compatibility Since 0.1

Password for trust store file.

Property ovsx.elasticsearch.relevance.rating
Type double
Default 1.0
Compatibility Since 0.1, deprecated in 0.2 (use ovsx.search.relevance.rating)

Weight of user ratings for computing relevance. This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.elasticsearch.relevance.downloads
Type double
Default 1.0
Compatibility Since 0.1, deprecated in 0.2 (use ovsx.search.relevance.downloads)

Weight of download counts for computing relevance. This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.elasticsearch.relevance.timestamp
Type double
Default 1.0
Compatibility Since 0.1, deprecated in 0.2 (use ovsx.search.relevance.timestamp)

Weight of publishing timestamps for computing relevance (newer extensions are ranked higher). This has an impact on the order of search results when sortBy is set to relevance.

Property ovsx.elasticsearch.relevance.unverified
Type double
Default 0.5
Compatibility Since 0.1, deprecated in 0.2 (use ovsx.search.relevance.unverified)

Relevance factor for unverified extension versions. The combined relevance from the averageRating, downloadCount and timestamp criteria is multiplied with this value if the publisher of the extension is not a member of the extension's namespace or the namespace has no owner.

Database Search

Property ovsx.databasesearch.enabled
Type boolean
Default false
Compatibility Since 0.2

Whether to enable search functionality though DB queries. Cannot be used together with ovsx.elasticsearch.enabled.

File Storage

Property ovsx.storage.azure.service-endpoint
Type string
Default
Compatibility Since 0.1

Azure blob service endpoint URL (without parameters, must end with a slash). This is required in order to enable the Azure storage service. Example: https://openvsx.blob.core.windows.net/

Property ovsx.storage.azure.sas-token
Type string
Default
Compatibility Since 0.1

The full query string containing the Azure SAS (Shared Access Signature) token.

Property ovsx.storage.azure.blob-container
Type string
Default openvsx-resources
Compatibility Since 0.1

Name of the Azure blob container.

Property ovsx.storage.gcp.project-id
Type string
Default
Compatibility Since 0.1

GCP project id. This can be omitted if the GCP client is able to detect the project from the environment.

Property ovsx.storage.gcp.bucket-id
Type string
Default
Compatibility Since 0.1

GCP bucket id. This is required in order to enable the Google Cloud storage service. Note that in order to upload files you need to authenticate with the storage service, e.g. by putting service account credentials into a file and pointing the environment variable GOOGLE_APPLICATION_CREDENTIALS to that file. Unauthenticated access is possible when migrating from GCP to another storage provider.

Property ovsx.storage.primary-service
Type string
Default
Compatibility Since 0.1

Storage service to use if multiple are active (azure-blob or google-cloud). All files that are not in the primary service are automatically migrated on application startup.

Property ovsx.storage.external-resource-types
Type string[]
Default *
Compatibility Since 0.1

Resource types to store in an external storage provider, or * for all types. Possible values are download (i.e. the vsix file), manifest, icon, readme, license and changelog. If only a subset of those types is chosen, the remaining types are stored as byte arrays in the database.

Property ovsx.storage.migration-delay
Type long
Default 500
Compatibility Since 0.1

Delay in milliseconds between storage type migration of each file. This delay is important to avoid excessive load in the server application, since migration is performed by the server itself on startup. Longer delays decrease server load, but increase the total duration of file migration.

Upstream Registry

Property ovsx.upstream.url
Type string
Default
Compatibility Since 0.1

Base URL of the upstream registry instance.

VS Code

Property ovsx.vscode.upstream.gallery-url
Type string
Default
Compatibility Since 0.1

Gallery URL of a registry instance from which to fetch extension UUIDs. These UUIDs are required by VS Code to identify and auto-update installed extensions. If no upstream gallery is set, random UUIDs are generated for all published extensions.

Eclipse

Property ovsx.eclipse.base-url
Type string
Default
Compatibility Since 0.1

Base URL of the Eclipse API.

Property ovsx.eclipse.publisher-agreement.version
Type string
Default
Compatibility Since 0.1

Current version of the Eclipse Publisher Agreement.

Property ovsx.eclipse.publisher-agreement.timezone
Type string
Default
Compatibility Since 0.1

java.time.ZoneId for timestamps returned by the Eclipse API.

Adding the Web UI

The Docker image of the server application does not include the web UI. The reason for this is that the UI can be customized. In case you don't need customization, you can use the default web UI image and deploy it next to the server.

Customization of the UI is done by creating an npm package with a dependency on openvsx-webui, which is a library of React components. A minimal frontend app is shown in the following TypeScript-React (tsx) code.

import * as ReactDOM from 'react-dom';
import * as React from 'react';
import { BrowserRouter } from 'react-router-dom';
import { ThemeProvider } from '@material-ui/styles';
import { Main, ExtensionRegistryService } from 'openvsx-webui';

const App: React.FunctionComponent = () => {
    const theme = ...        // Define a Material UI theme
    const pageSettings = ... // Define page settings (see below)
    const service = new ExtensionRegistryService();
    return <ThemeProvider theme={theme}>
        <Main pageSettings={pageSettings} service={service} />
    </ThemeProvider>;
};

const node = document.getElementById('main');
ReactDOM.render(<BrowserRouter><App /></BrowserRouter>, node);

See the Material UI documentation and the default theme to learn how to define a theme.

The page settings schema is defined in this interface. It includes general settings as well as React components to be rendered in specific parts of the UI.

Upstream Registry Instance

It is possible to configure another Open VSX instance as upstream. This means that API requests to extensions that are not found locally are forwarded to the upstream instance. With this mechanism, you can keep selected extensions in your own (private) instance and delagate to another (public) instance for all other extensions. For example, you can use https://open-vsx.org/ as upstream instance to enable access to all public extensions, but still point users to your private instance.

Clone this wiki locally