Jupyterhub 部署_qq63d4bef95921c的技术博客_51CTO博客

DockerSpawner官方文档DockerSpawner

DockerSpawner的安装

接受经过身份验证的用户,并在 Docker 容器中为该用户生成一个笔记本服务器。

安装

安装jupyterhub

$ python3 -m pip install jupyterhub

安装dockerspawner

$ python3 -m pip install dockerspawner

选择DockerSpawner

创建配置文件之后-->jupyterhub --generate-config

在配置文件中选择使用DockerSpawner

c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'

数据持久化

使用 时DockerSpawner​,用户的主目录默认情况下不是持久的,因此需要进行一些配置才能实现此目的,除非该目录用于临时或演示 JupyterHub 部署。

可以通过docker映射的方式来实现持久化。在配置文件jupyterhub_config.py​中配置c.DockerSpawner.volumes​来指定映射位置。如下:

# Explicitly set notebook directory because we'll be mounting a host volume to
# it.  Most jupyter/docker-stacks *-notebook images run the Notebook server as
# user `jovyan`, and set the notebook directory to `/home/jovyan/work`.
# We follow the same convention.
notebook_dir = os.environ.get('DOCKER_NOTEBOOK_DIR') or '/home/jovyan/work'
c.DockerSpawner.notebook_dir = notebook_dir

# Mount the real user's Docker volume on the host to the notebook user's
# notebook directory in the container
c.DockerSpawner.volumes = { 'jupyterhub-user-{username}': notebook_dir }

内存限制

c.Spawner.mem_limit = '2G'

该值可以是整数(字节),也可以是带有“K”、“M”、“G”或“T”前缀的字符串。

选择或者构建Docker镜像

默认情况下,DockerSpawner 使用quay.io/jupyterhub/singleuser​带有适当标签的映像来固定正确版本的 JupyterHub,但您也可以构建自己的映像。

案例

jupyterhub-deploy-docker

案例部署

GitHub - jupyterhub/jupyterhub-deploy-docker: Reference deployment of JupyterHub with docker

jupyterhub-deploy-docker关键组件

  • Host : 在主机上的 Docker 容器中运行JupyterHub 组件。

  • Authenticator:使用Native Authenticator对用户进行身份验证。任何用户都可以注册。

  • Spawner:使用DockerSpawner 在同一主机上的单独 Docker 容器中生成单用户 Jupyter Notebook 服务器。

  • Hub 数据的持久化:将 JupyterHub 数据持久化到主机上的 Docker 卷中。

  • 用户笔记本目录的持久性:将用户笔记本目录保留在主机上的 Docker 卷中。

克隆项目

git clone https://github.com/jupyterhub/jupyterhub-deploy-docker.git

进入到basic-example目录下,并构建Docker镜像

docker compose build

用户使用的docker镜像为quay.io/jupyter/base-notebook Notebook 映像,所以需要拉取该镜像

docker pull quay.io/jupyter/base-notebook:latest

运行upyterhub容器

docker compose up -d

如果要停止容器

docker compose down

案例分析

首先分析docker-compose.yml​文件

version: "3"

services:
  hub:
    build:
      # 当前目录为上下文
      context: .
      # 使用如下的自己构建的Docker镜像
      dockerfile: Dockerfile.jupyterhub
      args:
        # 定义了传递给Dockerfile的参数为latest
        JUPYTERHUB_VERSION: latest
    restart: always
    image: jupyterhub
    container_name: jupyterhub
    networks:
      - jupyterhub-network
    volumes:
      # 将主机上的"jupyterhub_config.py"文件绑定到容器内的"/srv/jupyterhub/jupyterhub_config.py"路径,并设置为只读模式。
      - "./jupyterhub_config.py:/srv/jupyterhub/jupyterhub_config.py:ro"
      # 将主机上的Docker套接字绑定到容器内的相同路径,并设置为读写模式。目的是可以在容器内部使用docker
      - "/var/run/docker.sock:/var/run/docker.sock:rw"
      # 将名为"jupyterhub-data"的卷绑定到容器内的"/data"路径
      - "jupyterhub-data:/data"
    ports:
      - "8000:8000"
    # 设置环境变量
    environment:
      # This username will be a JupyterHub admin
      JUPYTERHUB_ADMIN: admin
      # All containers will join this network
      DOCKER_NETWORK_NAME: jupyterhub-network
      # JupyterHub will spawn this Notebook image for users
      DOCKER_NOTEBOOK_IMAGE: quay.io/jupyter/base-notebook:latest
      # Notebook directory inside user image
      DOCKER_NOTEBOOK_DIR: /home/jovyan/work

volumes:
  jupyterhub-data:

networks:
  jupyterhub-network:
    name: jupyterhub-network

Dockerfile文件如下:

# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
ARG JUPYTERHUB_VERSION
FROM quay.io/jupyterhub/jupyterhub:$JUPYTERHUB_VERSION

# Install dockerspawner, nativeauthenticator
# hadolint ignore=DL3013
RUN python3 -m pip install --no-cache-dir \
    dockerspawner \
    	jupyterhub-nativeauthenticator

CMD ["jupyterhub", "-f", "/srv/jupyterhub/jupyterhub_config.py"]

创建了jupyterhub镜像,并在其中安装了dockerspawner和jupyterhub-nativeauthenticator,并创建了配置文件/srv/jupyterhub/jupyterhub_config.py

然后就是分析其jupyterhub的配置文件了

# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.

# Configuration file for JupyterHub
import os

c = get_config()  # noqa: F821

# We rely on environment variables to configure JupyterHub so that we
# avoid having to rebuild the JupyterHub container every time we change a
# configuration parameter.

# Spawn single-user servers as Docker containers
c.JupyterHub.spawner_class = "dockerspawner.DockerSpawner"

# Spawn containers from this image
c.DockerSpawner.image = os.environ["DOCKER_NOTEBOOK_IMAGE"]

# Connect containers to this Docker network
network_name = os.environ["DOCKER_NETWORK_NAME"]
c.DockerSpawner.use_internal_ip = True
c.DockerSpawner.network_name = network_name

# Explicitly set notebook directory because we'll be mounting a volume to it.
# Most `jupyter/docker-stacks` *-notebook images run the Notebook server as
# user `jovyan`, and set the notebook directory to `/home/jovyan/work`.
# We follow the same convention.
notebook_dir = os.environ.get("DOCKER_NOTEBOOK_DIR", "/home/jovyan/work")
c.DockerSpawner.notebook_dir = notebook_dir

# Mount the real user's Docker volume on the host to the notebook user's
# notebook directory in the container
c.DockerSpawner.volumes = {"jupyterhub-user-{username}": notebook_dir}

# Remove containers once they are stopped
c.DockerSpawner.remove = True

# For debugging arguments passed to spawned containers
c.DockerSpawner.debug = True

# User containers will access hub by container name on the Docker network
c.JupyterHub.hub_ip = "jupyterhub"
c.JupyterHub.hub_port = 8080

# Persist hub data on volume mounted inside container
c.JupyterHub.cookie_secret_file = "/data/jupyterhub_cookie_secret"
c.JupyterHub.db_url = "sqlite:////data/jupyterhub.sqlite"

# Authenticate users with Native Authenticator
c.JupyterHub.authenticator_class = "nativeauthenticator.NativeAuthenticator"

# Allow anyone to sign-up without approval
c.NativeAuthenticator.open_signup = True

# Allowed admins
admin = os.environ.get("JUPYTERHUB_ADMIN")
if admin:
    c.Authenticator.admin_users = [admin]