one grader multiple courses
bpfrd opened this issue · comments
Hello,
In the current nbgrader demos/demo_multiple_courses, you put the below code in grader-course101 home directory.
grader-course101 cat .jupyter/nbgrader_config.py
c = get_config()
c.CourseDirectory.root = '/home/grader-course101/course101'
c.CourseDirectory.course_id = "course101"
what if there is one grader for multiple courses? is it supported? How can I write services in this demo in z2jh?
thanks.
best
Hey @bpfrd , I've already replied to your Jupyter Forum post. But in case anyone else encounters a similar problem in the future, I would like to share the z2jh + nbgrader Helm chart we created, which addresses most of the issues you may come across during a basic deployment of nbgrader.
https://github.com/CERIT-SC/nbgrader-k8s
PS: Feel free to create an Issue/PR, if you find something ;-)
Hi @KrKOo
Great work! Thank you very much for sharing it. I followed the instructions in the github repo to run jupyterhub on minikube. The hub deployment is pending. The logs for the pod is empty but here is the output of "kubectl describe". I'm new to k8s. Would you have any idea what the problem is? should I have changed the storage class for minikube?
best
kubectl describe pod/hub-699bf99b98-tcvlb
Name: hub-699bf99b98-tcvlb
Namespace: default
Priority: 0
Service Account: hub
Node:
Labels: app=jupyterhub
component=hub
hub.jupyter.org/network-access-proxy-api=true
hub.jupyter.org/network-access-proxy-http=true
hub.jupyter.org/network-access-singleuser=true
pod-template-hash=699bf99b98
release=nbgrader
Annotations: checksum/config-map: 29f301903c4a63e2828219d4556e94d8f3e5d8b41736d48251811308ff14430c
checksum/secret: 540300c90e277cb9ddf05d05135cbbb6d60e73fd0c36d267df92a69d0dd3f4ba
Status: Pending
SeccompProfile: RuntimeDefault
IP:
IPs:
Controlled By: ReplicaSet/hub-699bf99b98
Containers:
hub:
Image: cerit.io/hubs/nbgrader-hub:10-11-2023
Port: 8081/TCP
Host Port: 0/TCP
Args:
jupyterhub
--config
/usr/local/etc/jupyterhub/jupyterhub_config.py
--debug
--upgrade-db
Limits:
cpu: 2
memory: 4Gi
Requests:
cpu: 2
memory: 4Gi
Liveness: http-get http://:http/hub/health delay=10s timeout=10s period=10s #success=1 #failure=10
Readiness: http-get http://:http/hub/health delay=10s timeout=10s period=10s #success=1 #failure=10
Environment:
PYTHONUNBUFFERED: 1
HELM_RELEASE_NAME: nbgrader
POD_NAMESPACE: default (v1:metadata.namespace)
CONFIGPROXY_AUTH_TOKEN: <set to the key 'hub.config.ConfigurableHTTPProxy.auth_token' in secret 'hub'> Optional: false
Mounts:
/etc/jupyter/ from nbgrader-config-global (ro)
/mnt/exchange from nbgrader-exchange (rw)
/srv/jupyterhub from pvc (rw)
/usr/local/etc/jupyterhub/config/ from config (rw)
/usr/local/etc/jupyterhub/jupyterhub_config.py from config (rw,path="jupyterhub_config.py")
/usr/local/etc/jupyterhub/secret/ from secret (rw)
/usr/local/etc/jupyterhub/z2jh.py from config (rw,path="z2jh.py")
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rbsjc (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: hub
Optional: false
secret:
Type: Secret (a volume populated by a Secret)
SecretName: hub
Optional: false
nbgrader-exchange:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: nbgrader-exchange
ReadOnly: false
nbgrader-config-global:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: nbgrader-config-global
Optional: false
pvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: hub-db-dir
ReadOnly: false
kube-api-access-rbsjc:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors:
Tolerations: hub.jupyter.org/dedicated=core:NoSchedule
hub.jupyter.org_dedicated=core:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
Warning FailedScheduling 2m13s (x2 over 7m29s) default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
PS C:\Users\bpfrd\nbgrader-k8s> kubectl get pod
NAME READY STATUS RESTARTS AGE
hub-699bf99b98-tcvlb 0/1 Pending 0 8m47s
proxy-5998987fc8-dc9dx 1/1 Running 0 8m47s
PS C:\Users\bpfrd\nbgrader-k8s> kubectl logs pod/hub-699bf99b98-tcvlb
Yes, if you are using minikube, you should set the "storageClassName" to "standard" (replace every instance of "nfs-csi" in the values.yaml file).
I guess it would be better to make the chart use the default storage class when none is set, but until then just use "standard" which is the default for minikube.
Thank you very much. It works now, but there seems to be a problem with nbgrader. When I try to get course1 as instructor1 I get the below error:
503 : Service Unavailable
Your server appears to be down. Try restarting it from the hub
and here is the pod log for jupyter-instructor1
[I 2023-12-07 19:20:43.713 ServerApp] 200 GET /user/instructor1/nbgrader_version?version=0.9.1&1701976843698 (instructor1@::ffff:127.0.0.1) 1.21ms
[NbGrader | WARNING] Config option kernel_spec_manager_class not recognized by NbGrader.
[W 2023-12-07 19:20:43.720 ServerApp] Local formgrader does not seem to be running
[I 2023-12-07 19:20:43.737 ServerApp] 200 GET /user/instructor1/formgraders?1701976843699 (instructor1@::ffff:127.0.0.1) 23.06ms
[I 2023-12-07 19:20:44.254 ServerApp] 204 PUT /user/instructor1/lab/api/workspaces/auto-4?1701976844240 (instructor1@::ffff:127.0.0.1) 1.47ms
[I 2023-12-07 19:20:49.096 ServerApp] 200 GET /user/instructor1/api/kernels?1701976849087 (instructor1@::ffff:127.0.0.1) 2.18ms
[I 2023-12-07 19:20:49.107 ServerApp] 200 GET /user/instructor1/api/sessions?1701976849088 (instructor1@::ffff:127.0.0.1) 1.67ms
[I 2023-12-07 19:20:49.111 ServerApp] 200 GET /user/instructor1/api/terminals?1701976849089 (instructor1@::ffff:127.0.0.1) 1.46ms
I have another question: I see in the z2jh tutorial that we can give options for spawning in the singleuser.profileList. But in this case, students and instructors should get courses they are registered in as options. Should I do it in bootstrap_pre_spawn?
@bpfrd Nice, you just found an error in the chart 😉 It is fixed now, just pull and "helm upgrade".
Regarding the second question. Depending on your use case, maybe this could help you:
https://github.com/jupyterhub/jupyterhub/blob/HEAD/examples/spawn-form/jupyterhub_config.py
https://jupyterhub.readthedocs.io/en/stable/reference/spawners.html#spawner-options-form
In the _options_form_default() method you could fetch the groups of the user and generate the form based on that.
In options_from_form() you just parse the options from the form, and then in bootstrap_pre_spawn you can access these options by spawner.user_options.get('option_name'). Based on that you can then do whatever you need.
Thank you very much. Everything works fine now.
I just noticed that in the graphical interface of formgrader there is no autograde.
I could be wrong, but I don't think that that's even a thing in nbgrader. The autograde button is on the page, which comes after you click on the number of submissions.
sorry, it was my bad. the autograde is available under assignments/assignment_id not assignments.
I have a general question about jupyterhub which couldn’t really find an answer in the jupyterhub documentation. What is the hierarchy of functions that are called in order in jupyterhub when a user logs in and uses the system.
for example you assigned BaseHandler.get_accessible_services = get_accessible_services but when is this function or other functions called?
thank you very much in advance
I don't think there is an easy way to find these things. The technique used in the chart is called 'monkey patching', which means that some methods from the JupyterHub source are replaced with a modified version of them. You will probably have to read the source code of JupyterHub to understand how those methods work and when they are getting called.
Regarding the BaseHandler.get_accessible_services method. The patch that I have in the chart will probably not be needed with a newer version of JupyterHub, since this PR addresses the same issue.
Hello,
I get a permission denied error when trying to install it on another system. Is the error happening on the hub container? how can we fix it?
best
$kubectl get pod
NAME READY STATUS RESTARTS AGE
hub-7dd77cd747-29sd5 0/1 CrashLoopBackOff 3 (37s ago) 2m10s
proxy-bffbd5c54-5x62h 1/1 Running 0 2m10s
$kubectl logs pod/hub-7dd77cd747-29sd5
[D 2023-12-12 11:33:46.056 JupyterHub application:902] Looking for /usr/local/etc/jupyterhub/jupyterhub_config in /srv/jupyterhub
Loading /usr/local/etc/jupyterhub/secret/values.yaml
No config at /usr/local/etc/jupyterhub/existing-secret/values.yaml
Loading extra config: 00-extra-config
[E 2023-12-12 11:33:46.468 JupyterHub app:3382]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/jupyterhub/app.py", line 3379, in launch_instance_async
await self.initialize(argv)
File "/usr/local/lib/python3.11/site-packages/jupyterhub/app.py", line 2857, in initialize
self.load_config_file(self.config_file)
File "/usr/local/lib/python3.11/site-packages/traitlets/config/application.py", line 113, in inner
return method(app, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/traitlets/config/application.py", line 950, in load_config_file
for (config, fname) in self._load_config_files(
File "/usr/local/lib/python3.11/site-packages/traitlets/config/application.py", line 909, in _load_config_files
config = loader.load_config()
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/traitlets/config/loader.py", line 626, in load_config
self._read_file_as_dict()
File "/usr/local/lib/python3.11/site-packages/traitlets/config/loader.py", line 659, in _read_file_as_dict
exec(compile(f.read(), conf_filename, "exec"), namespace, namespace) # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/etc/jupyterhub/jupyterhub_config.py", line 497, in
exec(config_py)
File "", line 174, in
File "", line 215, in makedirs
File "", line 225, in makedirs
PermissionError: [Errno 13] Permission denied: '/mnt/exchange/course1'
[D 2023-12-12 11:33:46.471 JupyterHub application:1028] Exiting application: jupyterhub
update: the error in the previous message only happens when the minikube cluster has more than one nodes
@bpfrd maybe you should open an issue in https://github.com/CERIT-SC/nbgrader-k8s ?
I'm closing it since the initial question has been answered, and the other issues are not about nbgrader.
Feel free to reopen it if necessary.