Skip to content

Commit d37bccc

Browse files
vivian-rookvivian-rook
authored
Merge branch 'master' into lsof
2 parents 3d211b4 + bdeeac3 commit d37bccc

File tree

11 files changed

+78
-21
lines changed

11 files changed

+78
-21
lines changed

‎.github/workflows/jobber.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
name: build and push jobber
22

33
on:
4-
pull_request:
4+
pull_request_target:
55
paths:
66
- images/jobber/**
77

88
jobs:
99
build-and-push:
1010
name: build and push jobber
11-
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v1
11+
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v2
1212
with:
1313
imagename: jobber
1414
secrets:

‎.github/workflows/nbserve.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
name: build and push nbserve
22

33
on:
4-
pull_request:
4+
pull_request_target:
55
paths:
66
- images/nbserve/**
77

88
jobs:
99
build-and-push:
1010
name: build and push nbserve
11-
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v1
11+
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v2
1212
with:
1313
imagename: nbserve
1414
secrets:

‎.github/workflows/paws-hub.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
name: build and push paws-hub
22

33
on:
4-
pull_request:
4+
pull_request_target:
55
paths:
66
- images/paws-hub/**
77

88
jobs:
99
build-and-push:
1010
name: build and push paws-hub
11-
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v1
11+
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v2
1212
with:
1313
imagename: paws-hub
1414
secrets:

‎.github/workflows/renderer.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
name: build and push renderer
22

33
on:
4-
pull_request:
4+
pull_request_target:
55
paths:
66
- images/renderer/**
77

88
jobs:
99
build-and-push:
1010
name: build and push renderer
11-
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v1
11+
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v2
1212
with:
1313
imagename: renderer
1414
secrets:

‎.github/workflows/singleuser.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
name: build and push singleuser
22

33
on:
4-
pull_request:
4+
pull_request_target:
55
paths:
66
- images/singleuser/**
77

88

99
jobs:
1010
build-and-push:
1111
name: build and push singleuser
12-
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v1
12+
uses: toolforge/github-actions/.github/workflows/build-and-push.yaml@build-and-push-v2
1313
with:
1414
imagename: singleuser
1515
secrets:

‎.github/workflows/update-container-tags.yaml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name: update container tags
22

33
on:
4-
pull_request:
4+
pull_request_target:
55

66
jobs:
77
update-container-tags:
@@ -10,10 +10,14 @@ jobs:
1010
- uses: actions/checkout@v2
1111
with:
1212
token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
13+
repository: ${{ github.event.pull_request.head.repo.full_name }}
1314

1415
- name: git fetch
1516
run: |
1617
git fetch
18+
- name: git checkout
19+
run: |
20+
git checkout ${{ github.head_ref }}
1721
1822
# Github actions doesn't have much in the way of loops. Using a matrix results
1923
# in multiple separate jobs being run, which will try to overwrite one another.

‎README.md‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# PAWS
22

3-
PAWS: A Web Shell (PAWS) is a Jupyter notebooks deployment that has been customized to make interacting with Wikimedia wikis easier. It allows users to create and share documents that contain live code, visualizations such as graphs, rich text, etc. The user created notebooks are a powerful tool that enables data analysis and scientific research, and also transforms the way in which programmers write code - by enabling an exploratory environment with a quick feedback loop, and a low barrier for entry through it's easy to use graphical interface.
3+
PAWS: A Web Shell (PAWS) is a Jupyter notebooks deployment that has been customized to make interacting with Wikimedia wikis easier. It allows users to create and share documents that contain live code, visualizations such as graphs, rich text, etc. The user created notebooks are a powerful tool that enables data analysis and scientific research, and also transforms the way in which programmers write code - by enabling an exploratory environment with a quick feedback loop, and a low barrier for entry through it's easy to use graphical interface.
44

55
## Contributing
66

@@ -33,15 +33,26 @@ install the dependencies for the PAWS dev environment with these steps:
3333
The rest of the setup instructions will display on screen as long as the install is successful.
3434
Please refer to the helm documentation from there.
3535

36+
If you are experiencing issues with the installation, you can try changing the driver configuration in minikube: https://minikube.sigs.k8s.io/docs/drivers/
37+
38+
- First delete the current cluster:
39+
40+
`minikube delete`
41+
42+
- Start a new cluster with the driver you want to use (e.g. docker, virtualbox, hyperkit, etc.):
43+
44+
`minikube start --driver=docker --kubernetes-version=v1.20.11`
45+
46+
Another possible solution if minikube is acting weird might be to upgrade minikube, or even to
47+
increase the default memory:
48+
49+
`minikube config set memory 4096`
50+
3651
NOTE: By default the mariadb chart keeps a PersistentVolumeClaim around for its storage even after
3752
uninstall. If you intend on rebuilding your dev environment later, you will need to use all the same
3853
values for DB and DB passwords if you don't delete that claim and volume (and the data from your
3954
last wiki will be in there--which means you keep your oauth grant!). The PVC for mediawiki gets cleaned up on uninstall.
4055

41-
If minikube is acting weird, it might be worth it to upgrade minikube or even to
42-
increase the default memory:
43-
`minikube config set memory 4096`
44-
4556
#### Working with images
4657
There are 8 images that are part of PAWS, in particular in the images/ directory. If you start a dev environment, it will pull those images from quay.io by default, just like in Wikimedia Cloud Services. If you are making changes to the images and testing those locally, you'll need to build them and tag them for your local environment, possibly setting them in your local values file with the tags you set.
4758

@@ -76,7 +87,7 @@ A notebook can be turned into a public notebook by publishing a link to it. This
7687

7788
If you want to run the copy yourself, or do interactive changes, you must download the notebook and reupload on your own account. Downloading the raw format of the previous example can be done by adding format=raw to the previous example …revisions-sql.ipynb?format=raw. This download-reupload-process is somewhat awkward.
7889

79-
Note that a notebook will always be published, as the link can be guessed, so don't add any private information.
90+
Note that a notebook will always be published, as the link can be guessed, so don't add any private information.
8091

8192
### To know more about paws have a look at:
8293
https://wikitech.wikimedia.org/wiki/PAWS

‎images/singleuser/Dockerfile‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ RUN curl --silent --location --fail ${RSTUDIO_URL} > /tmp/rstudio.deb && \
9595
# Setup OpenRefine
9696
ENV OPENREFINE_DIR /srv/openrefine
9797
RUN mkdir -p ${OPENREFINE_DIR} && cd ${OPENREFINE_DIR} && \
98-
curl -L https://github.com/OpenRefine/OpenRefine/releases/download/3.4.1/openrefine-linux-3.4.1.tar.gz | tar xzf - --strip=1
98+
curl -L https://github.com/OpenRefine/OpenRefine/releases/download/3.5.0/openrefine-linux-3.5.0.tar.gz | tar xzf - --strip=1
9999
COPY proxies/openrefine-logo.svg ${OPENREFINE_DIR}/openrefine-logo.svg
100100

101101
# Machine-learning type stuff
@@ -143,7 +143,12 @@ RUN sed -i -e '/^R_LIBS_USER=/s/^/#/' /etc/R/Renviron && \
143143
ENV VENV_DIR /srv/paws
144144
RUN install -d -o ${NB_USER} -g ${NB_USER} ${VENV_DIR}
145145

146-
ENV PATH=/srv/paws/pwb:/srv/paws/bin:/srv/paws:$PATH
146+
ENV PATH=/srv/paws/pwb:/srv/paws/bin:/srv/paws:/srv/julia/bin:$PATH
147+
148+
# Install Julia directories and depot path
149+
ENV JULIA_DEPOT_PATH /srv/julia-depot/
150+
RUN install -d -o ${NB_USER} -g ${NB_USER} /srv/julia
151+
RUN install -d -o ${NB_USER} -g ${NB_USER} ${JULIA_DEPOT_PATH}
147152

148153
USER ${NB_USER}
149154
RUN python3 -m venv /srv/paws
@@ -200,4 +205,8 @@ COPY --chown=tools.paws:tools.paws hide_clusters_tab.css /home/paws/.jupyter/cus
200205
# Setup custom config as needed
201206
COPY jupyter_notebook_config.py /srv/paws/etc/jupyter/jupyter_notebook_config.py
202207

208+
# install julia and julia kernel
209+
COPY install-julia /tmp/install-julia
210+
RUN /tmp/install-julia
211+
203212
EXPOSE 8888

‎images/singleuser/install-extensions‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ pip install --no-cache-dir \
99
py-heat-magic \
1010
jupyter-resource-usage \
1111
git+https://github.com/toolforge/nbpawspublic@main \
12+
git+https://github.com/toolforge/labpawspublic@main \
1213

1314

1415
jupyter nbextension install --py nbpawspublic --sys-prefix

‎images/singleuser/install-julia‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
# from: https://github.com/berkeley-dsep-infra/datahub/blob/staging/deployments/julia/image/install-julia.bash
4+
set -euo pipefail
5+
6+
export JULIA_PATH=/srv/julia/
7+
export JUPYTER_DATA_DIR=${VENV_DIR}/share/jupyter/
8+
9+
curl --silent --location --fail https://julialang-s3.julialang.org/bin/linux/x64/1.6/julia-1.6.3-linux-x86_64.tar.gz | tar xvz -C ${JULIA_PATH} --strip-components=1
10+
julia -e 'using Pkg; Pkg.add("IJulia"); Pkg.build("IJulia"); using IJulia; installkernel("Julia");'

0 commit comments

Comments
 (0)