feat: upgrade to Palm

This commit is contained in:
Muhammad Faraz Maqsood 2023-05-18 13:47:48 +05:00 committed by Régis Behmo
parent 2180f5c8e1
commit 61111bb128
11 changed files with 214 additions and 142 deletions

17
.github/workflows/sync.yml vendored Normal file
View File

@ -0,0 +1,17 @@
name: Sync with private repo
on:
push:
branches: [ master ]
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Add remote
run: git remote add overhangio https://${{ secrets.GIT_USERNAME }}:${{ secrets.GIT_PASSWORD }}@git.overhang.io/core/tutor-credentials.git
- name: Push
run: git push overhangio $GITHUB_REF

11
.gitlab-ci.yml Normal file
View File

@ -0,0 +1,11 @@
variables:
TUTOR_PLUGIN: credentials
TUTOR_IMAGES: credentials
TUTOR_PYPI_PACKAGE: tutor-credentials
OPENEDX_RELEASE: palm
GITHUB_REPO: overhangio/tutor-credentials
TUTOR_EXTRA_ENABLED_PLUGINS: discovery mfe
include:
- project: 'community/tutor-ci'
file: 'plugin-gitlab-ci.yml'

View File

@ -19,6 +19,13 @@ instructions, because git commits are used to generate release notes:
<!-- scriv-insert-here --> <!-- scriv-insert-here -->
<a id='changelog-16.0.0'></a>
## v16.0.0 (2023-06-15)
- [Improvement] Add a scriv-compliant changelog. (by @regisb)
- 💥[Feature] Upgrade to Palm. (by @Faraz32123)
## Version 14.0.0 (2022-09-15) ## Version 14.0.0 (2022-09-15)
* general production release for nutmeg.master * general production release for nutmeg.master

View File

@ -1,41 +1,40 @@
credentials plugin for `Tutor <https://docs.tutor.overhang.io>`__ Credentials plugin for `Tutor <https://docs.tutor.overhang.io>`__
=================================================================================== ===================================================================================
This is a plugin for `Tutor <https://docs.tutor.overhang.io>`_ that integrates the `Credentials <https://github.com/openedx/certificates/>`__ application in an Open edX platform. This is a plugin for `Tutor <https://docs.tutor.overhang.io>`_ that integrates the `Credentials <https://github.com/openedx/credentials/>`__ application in an Open edX platform.
This plugin also syncs the credentials database core_user table to openedx.auth_user, so after installing you should be able to authenticate with the same credentials that you use for your lms. Credentials application supports course and program certificates. This plugin offers an admin panel where user can do configurations for the certificates of his course and program.
Note that user will have to create the course/program using `Discovery plugin <https://github.com/overhangio/tutor-discovery>`__. Then Credentials plugin will be used for certificates configurations.
.. image:: https://github.com/overhangio/tutor-credentials/blob/main/doc/django-admin-screen-shot.png
:alt: Django Admin
Installation Installation
------------ ------------
:: ::
pip install git+https://github.com/lpm0073/tutor-contrib-credentials pip install https://github.com/overhangio/tutor-credentials.git
This plugin requires tutor>=12.0.0, the `Discovery plugin <https://github.com/overhangio/tutor-discovery>`__ and the `MFE plugin <https://github.com/overhangio/tutor-mfe>`__. If you have installed Tutor by downloading the pre-compiled binary, then both plugins should be automatically installed. You can confirm by running:: Note that this plugin is compatible with `Kubernetes integration <http://docs.tutor.overhang.io/k8s.html>`__.
tutor plugins list
Then, in any case you need to enable the plugins::
tutor plugins enable discovery mfe credentials
Services will have to be re-configured and restarted, so you are probably better off just running quickstart again::
tutor local quickstart
Note that this plugins is compatible with `Kubernetes integration <http://docs.tutor.overhang.io/k8s.html>`__. When deploying to a Kubernetes cluster run instead, noting that you'll need to create a public remote repository (ie AWS ECR)::
tutor plugins enable discovery mfe credentials
tutor config save --set CREDENTIALS_DOCKER_IMAGE=URI_OF_YOUR_REPOSITORY
tutor images build credentials
tutor images push credentials
docker tag YOUR-IMAGE-NAME YOUR-IMAGE-NAME:latest
docker push YOUR-IMAGE-NAME:latest
tutor k8s quickstart
For further instructions on how to setup Credentials with Open edX, check the `Official Credentials documentation <https://readthedocs.org/projects/edx-credentials/>`__. For further instructions on how to setup Credentials with Open edX, check the `Official Credentials documentation <https://readthedocs.org/projects/edx-credentials/>`__.
Usage
-----
::
pip install tutor-credentials
tutor plugins enable discovery mfe credentials
tutor local launch
Using Django Admin
~~~~~~~~~~~~~~~~~~
The credentials user interface will be available at http://credentials.local.overhang.io for a local instance, and at ``CREDENTIALS_HOST`` (by default: ``http(s)://credentials.<your lms host>``) in production. In order to run commands from the UI login with an admin user at: http://credentials.local.overhang.io/admin/. User should be able to authenticate with the same username and password that he used for his lms.
Configuration Configuration
------------- -------------
@ -47,7 +46,7 @@ Application configuration
- ``CREDENTIALS_MYSQL_USERNAME`` (default: ``"credentials"``) - ``CREDENTIALS_MYSQL_USERNAME`` (default: ``"credentials"``)
- ``CREDENTIALS_MYSQL_PASSWORD`` (default: ``"{{ 8|random_string }}"``) - ``CREDENTIALS_MYSQL_PASSWORD`` (default: ``"{{ 8|random_string }}"``)
- ``CREDENTIALS_CATALOG_API_URL`` (default: ``"{{ LMS_HOST }}"``) - ``CREDENTIALS_CATALOG_API_URL`` (default: ``"{{ LMS_HOST }}"``)
- ``CREDENTIALS_DOCKER_IMAGE`` (default: ``"{{ DOCKER_REGISTRY }}lpm0073/openedx-credentials:{{ CREDENTIALS_VERSION }}"``) - ``CREDENTIALS_DOCKER_IMAGE`` (default: ``"{{ DOCKER_REGISTRY }}overhangio/openedx-credentials:{{ CREDENTIALS_VERSION }}"``)
- ``CREDENTIALS_EXTRA_PIP_REQUIREMENTS`` (default: ``[]``) - ``CREDENTIALS_EXTRA_PIP_REQUIREMENTS`` (default: ``[]``)
- ``CREDENTIALS_PRIVACY_POLICY_URL`` (default: ``"LMS_HOST/pricacy-policy"``) - ``CREDENTIALS_PRIVACY_POLICY_URL`` (default: ``"LMS_HOST/pricacy-policy"``)
- ``CREDENTIALS_SITE_NAME`` (default: ``"LMS_HOST"``) - ``CREDENTIALS_SITE_NAME`` (default: ``"LMS_HOST"``)
@ -68,7 +67,7 @@ Marketing & Theming
- ``CREDENTIALS_FAVICON_URL`` (default: ``"https://edx-cdn.org/v3/default/favicon.ico"``) - ``CREDENTIALS_FAVICON_URL`` (default: ``"https://edx-cdn.org/v3/default/favicon.ico"``)
- ``CREDENTIALS_THEME_NAME`` (default: ``"edx-theme"``) - ``CREDENTIALS_THEME_NAME`` (default: ``"edx-theme"``)
Back end authentication Backend authentication
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
- ``CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_KEY`` (default: ``"credentials-backend-service-key"``) - ``CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_KEY`` (default: ``"credentials-backend-service-key"``)
@ -87,18 +86,6 @@ Application Third party authentication
- ``CREDENTIALS_SOCIAL_AUTH_EDX_OAUTH2_SECRET`` (default: ``"credentials-sso-secret"``) - ``CREDENTIALS_SOCIAL_AUTH_EDX_OAUTH2_SECRET`` (default: ``"credentials-sso-secret"``)
- ``CREDENTIALS_SOCIAL_AUTH_EDX_OAUTH2_LOGOUT_URL`` (default: ``"{{ SOCIAL_AUTH_EDX_OAUTH2_ISSUER }}/logout"``) - ``CREDENTIALS_SOCIAL_AUTH_EDX_OAUTH2_LOGOUT_URL`` (default: ``"{{ SOCIAL_AUTH_EDX_OAUTH2_ISSUER }}/logout"``)
Operations
----------
.. image:: https://github.com/lpm0073/tutor-contrib-credentials/blob/main/doc/django-admin-screen-shot.png
:alt: Django Admin
Using Django Admin
~~~~~~~~~~~~~~~~~~
The credentials user interface will be available at http://credentials.local.overhang.io for a local instance, and at ``CREDENTIALS_HOST`` (by default: ``http(s)://credentials.<your lms host>``) in production. In order to run commands from the UI login with an admin user at: http://credentials.local.overhang.io/admin/
Funding Funding
------- -------
@ -106,7 +93,7 @@ Funding
:alt: Academia Central :alt: Academia Central
:target: https://www.academiacentral.org/ :target: https://www.academiacentral.org/
This plugin was developed and open sourced to the community thanks to the generous support of `Academia Central <https://www.academiacentral.org/>`_. Thank you! This plugin was initially developed and open sourced to the community thanks to the generous support of `Academia Central <https://www.academiacentral.org/>`_. Thank you!
License License
------- -------

View File

@ -1 +0,0 @@
- [Improvement] Add a scriv-compliant changelog. (by @regisb)

View File

@ -25,12 +25,12 @@ ABOUT = load_about()
setup( setup(
name="tutor-contrib-credentials", name="tutor-credentials",
version=ABOUT["__version__"], version=ABOUT["__version__"],
url="https://github.com/lpm0073/tutor-contrib-credentials", url="https://github.com/overhangio/tutor-credentials.git",
project_urls={ project_urls={
"Code": "https://github.com/lpm0073/tutor-contrib-credentials", "Code": "https://github.com/overhangio/tutor-credentials.git",
"Issue tracker": "https://github.com/lpm0073/tutor-contrib-credentials/issues", "Issue tracker": "https://github.com/overhangio/tutor-credentials.git/issues",
"Community": "https://discuss.overhang.io", "Community": "https://discuss.overhang.io",
}, },
license="AGPLv3", license="AGPLv3",
@ -40,22 +40,18 @@ setup(
long_description=load_readme(), long_description=load_readme(),
packages=find_packages(exclude=["tests*"]), packages=find_packages(exclude=["tests*"]),
include_package_data=True, include_package_data=True,
python_requires=">=3.7", python_requires=">=3.8",
install_requires=["tutor"], install_requires=["tutor>=16.0.0,<17.0.0", "tutor-discovery>=16.0.0,<17.0.0", "tutor-mfe>=16.0.0,<17.0.0"],
entry_points={ entry_points={"tutor.plugin.v1": ["credentials = tutorcredentials.plugin"]},
"tutor.plugin.v1": [
"credentials = tutorcredentials.plugin"
]
},
classifiers=[ classifiers=[
"Development Status :: 3 - Alpha", "Development Status :: 3 - Alpha",
"Intended Audience :: Developers", "Intended Audience :: Developers",
"License :: OSI Approved :: GNU Affero General Public License v3", "License :: OSI Approved :: GNU Affero General Public License v3",
"Operating System :: OS Independent", "Operating System :: OS Independent",
"Programming Language :: Python", "Programming Language :: Python",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
], ],
) )

View File

@ -1,7 +1,8 @@
__version__ = "14.0.0" __version__ = "16.0.0"
# Handle version suffix for nightly, just like tutor core. # Handle version suffix for nightly, just like tutor core.
__version_suffix__ = "" __version_suffix__ = ""
if __version_suffix__: if __version_suffix__:
__version__ += "-" + __version_suffix__ __version__ += "-" + __version_suffix__

View File

@ -6,6 +6,6 @@ credentials:
volumes: volumes:
- ../plugins/credentials/apps/credentials/settings:/openedx/credentials/credentials/settings/tutor:ro - ../plugins/credentials/apps/credentials/settings:/openedx/credentials/credentials/settings/tutor:ro
depends_on: depends_on:
- lms
- discovery - discovery
{% if RUN_MYSQL %}- mysql{% endif %} {% if RUN_MYSQL %}- mysql{% endif %}
{% if RUN_LMS %}- lms{% endif %}

View File

@ -1,8 +1,11 @@
from __future__ import annotations
from glob import glob from glob import glob
import os import os
import pkg_resources import pkg_resources
import typing as t
from tutor import hooks from tutor import hooks as tutor_hooks
from .__about__ import __version__ from .__about__ import __version__
@ -11,17 +14,16 @@ from .__about__ import __version__
# CONFIGURATION # CONFIGURATION
######################################## ########################################
hooks.Filters.CONFIG_DEFAULTS.add_items( tutor_hooks.Filters.CONFIG_DEFAULTS.add_items(
[ [
# Add your new settings that have default values here. # Add your new settings that have default values here.
# Each new setting is a pair, (setting_name, default_value). # Each new setting is a pair, (setting_name, default_value).
# Prefix your setting names with 'CREDENTIALS_'. # Prefix your setting names with 'CREDENTIALS_'.
("CREDENTIALS_VERSION", __version__), ("CREDENTIALS_VERSION", __version__),
("CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_PROVIDER_URL", "http://lms:8000/oauth2"), ("CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_PROVIDER_URL", "http://lms:8000/oauth2"),
("CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_KEY", "{{ CREDENTIALS_OAUTH2_KEY }}"), ("CREDENTIALS_BACKEND_SERVICE_EDX_OAUTH2_KEY", "{{ CREDENTIALS_OAUTH2_KEY }}"),
("CREDENTIALS_CATALOG_API_URL", "{{ LMS_HOST }}"), ("CREDENTIALS_CATALOG_API_URL", "{{ LMS_HOST }}"),
("CREDENTIALS_DOCKER_IMAGE", "{{ DOCKER_REGISTRY }}lpm0073/openedx-credentials:{{ CREDENTIALS_VERSION }}"), ("CREDENTIALS_DOCKER_IMAGE", "{{ DOCKER_REGISTRY }}overhangio/openedx-credentials:{{ CREDENTIALS_VERSION }}"),
("CREDENTIALS_EXTRA_PIP_REQUIREMENTS", []), ("CREDENTIALS_EXTRA_PIP_REQUIREMENTS", []),
("CREDENTIALS_FAVICON_URL", "https://edx-cdn.org/v3/default/favicon.ico"), ("CREDENTIALS_FAVICON_URL", "https://edx-cdn.org/v3/default/favicon.ico"),
("CREDENTIALS_HOST", "credentials.{{ LMS_HOST }}"), ("CREDENTIALS_HOST", "credentials.{{ LMS_HOST }}"),
@ -55,7 +57,7 @@ hooks.Filters.CONFIG_DEFAULTS.add_items(
] ]
) )
hooks.Filters.CONFIG_UNIQUE.add_items( tutor_hooks.Filters.CONFIG_UNIQUE.add_items(
[ [
# Add settings that don't have a reasonable default for all users here. # Add settings that don't have a reasonable default for all users here.
# For instance, passwords, secret keys, etc. # For instance, passwords, secret keys, etc.
@ -72,7 +74,7 @@ hooks.Filters.CONFIG_UNIQUE.add_items(
] ]
) )
hooks.Filters.CONFIG_OVERRIDES.add_items( tutor_hooks.Filters.CONFIG_OVERRIDES.add_items(
[ [
# Danger zone! # Danger zone!
# Add values to override settings from Tutor core or other plugins here. # Add values to override settings from Tutor core or other plugins here.
@ -87,48 +89,83 @@ hooks.Filters.CONFIG_OVERRIDES.add_items(
######################################## ########################################
# To run the script from templates/credentials/tasks/myservice/init, add: # To run the script from templates/credentials/tasks/myservice/init, add:
hooks.Filters.COMMANDS_INIT.add_item(( MY_INIT_TASKS = [
"mysql", ("mysql", ("templates", "credentials", "tasks", "mysql", "init")),
("credentials", "tasks", "mysql", "init"), ("lms", ("templates", "credentials", "tasks", "lms", "init")),
)) ("credentials", ("templates", "credentials", "tasks", "credentials", "init")),
hooks.Filters.COMMANDS_INIT.add_item(( ("mysql", ("templates", "credentials", "tasks", "mysql", "sync_users")),
"lms", ]
("credentials", "tasks", "lms", "init"),
)) HERE = os.path.abspath(os.path.dirname(__file__))
hooks.Filters.COMMANDS_INIT.add_item(( for service, template_path in MY_INIT_TASKS:
"credentials", full_path: str = os.path.join(HERE, *template_path)
("credentials", "tasks", "credentials", "init"),
)) with open(full_path, encoding="utf-8") as init_task_file:
hooks.Filters.IMAGES_BUILD.add_item(( init_task: str = init_task_file.read()
"credentials", tutor_hooks.Filters.CLI_DO_INIT_TASKS.add_item((service, init_task))
("plugins", "credentials", "build", "credentials"),
"{{ CREDENTIALS_DOCKER_IMAGE }}", ########################################
(), # Credentials Public Host
)) ########################################
hooks.Filters.COMMANDS_INIT.add_item((
"mysql",
("credentials", "tasks", "mysql", "sync_users"), @tutor_hooks.Filters.APP_PUBLIC_HOSTS.add()
)) def _print_credentials_public_hosts(hosts: list[str], context_name: t.Literal["local", "dev"]) -> list[str]:
if context_name == "dev":
hosts += ["{{ CREDENTIALS_HOST }}:8150"]
else:
hosts += ["{{ CREDENTIALS_HOST }}"]
return hosts
########################################
# Mount Credentials
########################################
REPO_NAME = "credentials"
# Automount /openedx/credentials folder from the container
@tutor_hooks.Filters.COMPOSE_MOUNTS.add()
def _mount_credentials_apps(mounts, path_basename):
if path_basename == REPO_NAME:
app_name = REPO_NAME
mounts += [(app_name, "/openedx/credentials")]
return mounts
# Bind-mount repo at build-time, both for prod and dev images
@tutor_hooks.Filters.IMAGES_BUILD_MOUNTS.add()
def _mount_credentials_on_build(mounts: list[tuple[str, str]], host_path: str) -> list[tuple[str, str]]:
path_basename = os.path.basename(host_path)
if path_basename == REPO_NAME:
app_name = REPO_NAME
mounts.append((app_name, f"{app_name}-src"))
mounts.append((f"{app_name}-dev", f"{app_name}-src"))
return mounts
######################################## ########################################
# DOCKER IMAGE MANAGEMENT # DOCKER IMAGE MANAGEMENT
######################################## ########################################
# To build an image with `tutor images build myimage`, add a Dockerfile to templates/credentials/build/myimage and write: # To build an image with `tutor images build myimage`, add a Dockerfile to templates/credentials/build/myimage and write:
hooks.Filters.IMAGES_BUILD.add_item(( tutor_hooks.Filters.IMAGES_BUILD.add_item(
"credentials", (
("plugins", "credentials", "build", "credentials"), "credentials",
"{{ CREDENTIALS_DOCKER_IMAGE }}", ("plugins", "credentials", "build", "credentials"),
(), "{{ CREDENTIALS_DOCKER_IMAGE }}",
)) (),
)
)
# To pull/push an image with `tutor images pull myimage` and `tutor images push myimage`, write: # To pull/push an image with `tutor images pull myimage` and `tutor images push myimage`, write:
# hooks.Filters.IMAGES_PULL.add_item(( # tutor_hooks.Filters.IMAGES_PULL.add_item((
# "myimage", # "myimage",
# "docker.io/myimage:{{ CREDENTIALS_VERSION }}", # "docker.io/myimage:{{ CREDENTIALS_VERSION }}",
# ) # )
# hooks.Filters.IMAGES_PUSH.add_item(( # tutor_hooks.Filters.IMAGES_PUSH.add_item((
# "myimage", # "myimage",
# "docker.io/myimage:{{ CREDENTIALS_VERSION }}", # "docker.io/myimage:{{ CREDENTIALS_VERSION }}",
# ) # )
@ -140,14 +177,14 @@ hooks.Filters.IMAGES_BUILD.add_item((
# this section as-is :) # this section as-is :)
######################################## ########################################
hooks.Filters.ENV_TEMPLATE_ROOTS.add_items( tutor_hooks.Filters.ENV_TEMPLATE_ROOTS.add_items(
# Root paths for template files, relative to the project root. # Root paths for template files, relative to the project root.
[ [
pkg_resources.resource_filename("tutorcredentials", "templates"), pkg_resources.resource_filename("tutorcredentials", "templates"),
] ]
) )
hooks.Filters.ENV_TEMPLATE_TARGETS.add_items( tutor_hooks.Filters.ENV_TEMPLATE_TARGETS.add_items(
# For each pair (source_path, destination_path): # For each pair (source_path, destination_path):
# templates at ``source_path`` (relative to your ENV_TEMPLATE_ROOTS) will be # templates at ``source_path`` (relative to your ENV_TEMPLATE_ROOTS) will be
# rendered to ``destination_path`` (relative to your Tutor environment). # rendered to ``destination_path`` (relative to your Tutor environment).
@ -173,4 +210,4 @@ for path in glob(
) )
): ):
with open(path, encoding="utf-8") as patch_file: with open(path, encoding="utf-8") as patch_file:
hooks.Filters.ENV_PATCHES.add_item((os.path.basename(path), patch_file.read())) tutor_hooks.Filters.ENV_PATCHES.add_item((os.path.basename(path), patch_file.read()))

View File

@ -0,0 +1,12 @@
from credentials.settings.devstack import * # pylint: disable=wildcard-import, unused-wildcard-import
{% include "credentials/apps/credentials/settings/partials/common.py" %}
SOCIAL_AUTH_EDX_OAUTH2_PUBLIC_URL_ROOT = "{% if ENABLE_HTTPS %}https{% else %}http{% endif %}://{{ LMS_HOST }}:8000"
BACKEND_SERVICE_EDX_OAUTH2_KEY = "{{ CREDENTIALS_OAUTH2_KEY }}"
# Disable API caching, which makes it a pain to troubleshoot issues
USE_API_CACHING = False
{{ patch("credentials-settings-development") }}

View File

@ -1,12 +1,14 @@
{% if is_buildkit_enabled() %}# syntax=docker/dockerfile:1.4{% endif %}
###### Minimal image with base system requirements for most stages
FROM docker.io/ubuntu:20.04 as minimal FROM docker.io/ubuntu:20.04 as minimal
LABEL maintainer="Lawrence McDaniel <lpm0073@gmail.com>" LABEL maintainer="Lawrence McDaniel <lpm0073@gmail.com>"
ENV DEBIAN_FRONTEND=noninteractive ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && \ RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked{% endif %} \
apt update && \
apt install -y build-essential curl git language-pack-en apt install -y build-essential curl git language-pack-en
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8 ENV LC_ALL en_US.UTF-8
{{ patch("credentials-dockerfile-minimal") }} {{ patch("credentials-dockerfile-minimal") }}
@ -14,29 +16,29 @@ ENV LC_ALL en_US.UTF-8
###### Install python with pyenv in /opt/pyenv and create virtualenv in /openedx/venv ###### Install python with pyenv in /opt/pyenv and create virtualenv in /openedx/venv
FROM minimal as python FROM minimal as python
# https://github.com/pyenv/pyenv/wiki/Common-build-problems#prerequisites # https://github.com/pyenv/pyenv/wiki/Common-build-problems#prerequisites
RUN apt update && \ RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked {% endif %}apt update && \
apt install -y libssl-dev zlib1g-dev libbz2-dev \ apt install -y libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \ libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
ARG PYTHON_VERSION=3.8.12
ENV PYENV_ROOT /opt/pyenv
RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.2.2 --depth 1
RUN $PYENV_ROOT/bin/pyenv install $PYTHON_VERSION
RUN $PYENV_ROOT/versions/$PYTHON_VERSION/bin/python -m venv /openedx/venv
###### Install Dockerize to wait for mysql DB availability # Install pyenv
FROM minimal as dockerize # https://www.python.org/downloads/
# https://github.com/powerman/dockerize/releases # https://github.com/pyenv/pyenv/releases
ARG DOCKERIZE_VERSION=v0.16.0 ARG PYTHON_VERSION=3.8.15
RUN dockerize_url="https://github.com/powerman/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-$(uname -m | sed 's@aarch@arm@')" \ ENV PYENV_ROOT /opt/pyenv
&& echo "Downloading dockerize from $dockerize_url" \ RUN git clone https://github.com/pyenv/pyenv $PYENV_ROOT --branch v2.3.17 --depth 1
&& curl --fail --location --output /usr/local/bin/dockerize $dockerize_url \
&& chmod a+x /usr/local/bin/dockerize # Install Python
RUN $PYENV_ROOT/bin/pyenv install $PYTHON_VERSION
# Create virtualenv
RUN $PYENV_ROOT/versions/$PYTHON_VERSION/bin/python -m venv /openedx/venv
###### Checkout credentials ###### Checkout credentials
FROM minimal as code FROM minimal as code
ARG CREDENTIALS_REPOSITORY=https://github.com/edx/credentials.git ARG CREDENTIALS_REPOSITORY=https://github.com/edx/credentials.git
ARG CREDENTIALS_VERSION="open-release/nutmeg.master" ARG CREDENTIALS_VERSION="{{ OPENEDX_COMMON_VERSION }}"
RUN mkdir -p /openedx/credentials && \ RUN mkdir -p /openedx/credentials && \
git clone $CREDENTIALS_REPOSITORY --branch $CREDENTIALS_VERSION --depth 1 /openedx/credentials git clone $CREDENTIALS_REPOSITORY --branch $CREDENTIALS_VERSION --depth 1 /openedx/credentials
WORKDIR /openedx/credentials WORKDIR /openedx/credentials
@ -47,7 +49,7 @@ WORKDIR /openedx/credentials
# mcdaniel: including these just in case there are implied dependencies # mcdaniel: including these just in case there are implied dependencies
# on emitted credential data, which would be platform specific obviously. # on emitted credential data, which would be platform specific obviously.
FROM minimal as locales FROM minimal as locales
ARG OPENEDX_I18N_VERSION={{ OPENEDX_COMMON_VERSION }} ARG OPENEDX_I18N_VERSION="{{ OPENEDX_COMMON_VERSION }}"
RUN cd /tmp \ RUN cd /tmp \
&& curl -L -o openedx-i18n.tar.gz https://github.com/openedx/openedx-i18n/archive/$OPENEDX_I18N_VERSION.tar.gz \ && curl -L -o openedx-i18n.tar.gz https://github.com/openedx/openedx-i18n/archive/$OPENEDX_I18N_VERSION.tar.gz \
&& tar xzf /tmp/openedx-i18n.tar.gz \ && tar xzf /tmp/openedx-i18n.tar.gz \
@ -60,7 +62,9 @@ FROM python as python-requirements
ENV PATH /openedx/venv/bin:${PATH} ENV PATH /openedx/venv/bin:${PATH}
ENV VIRTUAL_ENV /openedx/venv/ ENV VIRTUAL_ENV /openedx/venv/
RUN apt update && apt install -y software-properties-common libmysqlclient-dev libxmlsec1-dev libgeos-dev RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked {% endif %}apt update \
&& apt install -y software-properties-common libmysqlclient-dev libxmlsec1-dev libgeos-dev
# Note that this means that we need to reinstall all requirements whenever there is a # Note that this means that we need to reinstall all requirements whenever there is a
# change in credentials, which sucks. But there is no obvious alternative, as we need # change in credentials, which sucks. But there is no obvious alternative, as we need
@ -69,26 +73,32 @@ COPY --from=code /openedx/credentials /openedx/credentials
WORKDIR /openedx/credentials WORKDIR /openedx/credentials
# Install the right version of pip/setuptools # Install the right version of pip/setuptools
RUN pip install setuptools==44.1.0 pip==20.0.2 wheel==0.34.2 RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}pip install \
# https://pypi.org/project/setuptools/
# https://pypi.org/project/pip/
# https://pypi.org/project/wheel/
setuptools==67.7.2 pip==23.1.2. wheel==0.40.0
# Install base requirements # Install base requirements
RUN pip install -r requirements/pip_tools.txt RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}pip install -r requirements/pip_tools.txt
RUN pip install -r requirements.txt RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}pip install -r requirements.txt
# Install django-redis for using redis as a django cache # Install extra requirements
RUN pip install django-redis==4.12.1 RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}pip install \
# Use redis as a django cache https://pypi.org/project/django-redis/
# Install uwsgi django-redis==5.2.0 \
RUN pip install uwsgi==2.0.20 # uwsgi server https://pypi.org/project/uWSGI/
uwsgi==2.0.21
{{ patch("credentials-dockerfile-post-python-requirements") }} {{ patch("credentials-dockerfile-post-python-requirements") }}
# Install private requirements: this is useful for installing custom xblocks.
COPY ./requirements/ /openedx/requirements COPY ./requirements/ /openedx/requirements
RUN cd /openedx/requirements/ \ RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}cd /openedx/requirements/ \
&& touch ./private.txt \ && touch ./private.txt \
&& pip install -r ./private.txt && pip install -r ./private.txt
{% for extra_requirement in CREDENTIALS_EXTRA_PIP_REQUIREMENTS %}RUN pip install {{ extra_requirement }} {% for extra_requirement in CREDENTIALS_EXTRA_PIP_REQUIREMENTS %}RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.cache/pip,sharing=shared {% endif %}pip install '{{ extra_requirements }}'
{% endfor %} {% endfor %}
###### Install nodejs with nodeenv in /openedx/nodeenv ###### Install nodejs with nodeenv in /openedx/nodeenv
@ -96,14 +106,15 @@ FROM python as nodejs-requirements
ENV PATH /openedx/nodeenv/bin:/openedx/venv/bin:${PATH} ENV PATH /openedx/nodeenv/bin:/openedx/venv/bin:${PATH}
# Install nodeenv with the version provided by credentials # Install nodeenv with the version provided by credentials
RUN pip install nodeenv==1.6.0 # https://github.com/pyenv/pyenv/releases
RUN nodeenv /openedx/nodeenv --node=12.13.0 --prebuilt RUN pip install nodeenv==1.7.0
RUN nodeenv /openedx/nodeenv --node=16.14.0 --prebuilt
# Install nodejs requirements # Install nodejs requirements
ARG NPM_REGISTRY={{ NPM_REGISTRY }} ARG NPM_REGISTRY='{{ NPM_REGISTRY }}'
COPY --from=code /openedx/credentials/package.json /openedx/credentials/package.json COPY --from=code /openedx/credentials/package.json /openedx/credentials/package.json
WORKDIR /openedx/credentials WORKDIR /openedx/credentials
RUN npm install --verbose --registry=$NPM_REGISTRY RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/openedx/.npm/,sharing=shared {% endif %}npm install --verbose --registry=$NPM_REGISTRY
###### Production image with system and python requirements ###### Production image with system and python requirements
FROM minimal as production FROM minimal as production
@ -111,24 +122,27 @@ FROM minimal as production
# Install system requirements # Install system requirements
# mcdaniel: these are the edx-platform system requirements. # mcdaniel: these are the edx-platform system requirements.
# TO DO: remove any packages that are not needed for Credentials. # TO DO: remove any packages that are not needed for Credentials.
RUN apt update && \ RUN {% if is_buildkit_enabled() %}--mount=type=cache,target=/var/cache/apt,sharing=locked \
apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng-dev libsqlite3-dev libxmlsec1-dev lynx ntp pkg-config rdfind && \ --mount=type=cache,target=/var/lib/apt,sharing=locked {% endif %}apt update \
rm -rf /var/lib/apt/lists/* && apt install -y gettext gfortran graphviz graphviz-dev libffi-dev libfreetype6-dev libgeos-dev libjpeg8-dev liblapack-dev libmysqlclient-dev libpng-dev libsqlite3-dev libxmlsec1-dev lynx mysql-client ntp pkg-config rdfind
# From then on, run as unprivileged "app" user # From then on, run as unprivileged "app" user
ARG APP_USER_ID=1000 ARG APP_USER_ID=1000
RUN if [ "$APP_USER_ID" = 0 ]; then echo "app user may not be root" && false; fi
RUN useradd --home-dir /openedx --create-home --shell /bin/bash --uid ${APP_USER_ID} app RUN useradd --home-dir /openedx --create-home --shell /bin/bash --uid ${APP_USER_ID} app
USER ${APP_USER_ID} USER ${APP_USER_ID}
# change file ownership to the new app user # change file ownership to the new app user
COPY --from=dockerize /usr/local/bin/dockerize /usr/local/bin/dockerize
COPY --chown=app:app --from=code /openedx/credentials /openedx/credentials COPY --chown=app:app --from=code /openedx/credentials /openedx/credentials
COPY --chown=app:app --from=locales /openedx/locale /openedx/locale COPY --chown=app:app --from=locales /openedx/locale /openedx/locale
COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv
COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv
COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements
COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv
COPY --chown=app:app --from=nodejs-requirements /openedx/credentials/node_modules /openedx/credentials/node_modules COPY --chown=app:app --from=nodejs-requirements /openedx/credentials/node_modules /openedx/node_modules
# Symlink node_modules such that we can bind-mount the credentials repository
RUN ln -s /openedx/node_modules /openedx/credentials/node_modules
ENV PATH /openedx/venv/bin:./node_modules/.bin:/openedx/nodeenv/bin:${PATH} ENV PATH /openedx/venv/bin:./node_modules/.bin:/openedx/nodeenv/bin:${PATH}
ENV VIRTUAL_ENV /openedx/venv/ ENV VIRTUAL_ENV /openedx/venv/
@ -141,15 +155,6 @@ RUN pip install -r requirements/production.in
RUN echo "{}" > /openedx/config.yml RUN echo "{}" > /openedx/config.yml
ENV CREDENTIALS_CFG /openedx/config.yml ENV CREDENTIALS_CFG /openedx/config.yml
# mcdaniel: this breaks. no idea if we really need it.
# -----------------------------------------------------------------------------
# Copy user-specific locales to /openedx/locale/user/locale and compile them
# RUN mkdir /openedx/locale/user
# COPY --chown=app:app ./locale/ /openedx/locale/user/locale/
# RUN cd /openedx/locale/user && \
# django-admin compilemessages -v1
# -----------------------------------------------------------------------------
{{ patch("credentials-dockerfile-pre-assets") }} {{ patch("credentials-dockerfile-pre-assets") }}
# Collect static assets # Collect static assets