What is this?

During the JupyterHub leadership meeting (North America) of June 2025, the breakout group comprising of Yuvi, Martha Cryan and Ryan Lovett came up with the following diagram to explain the aspirational mission of JupyterHub:

This document is Yuvi’s interpretation of one of the technosocial changes required to achieve this aspiration. It’s also heavily influenced by prior conversations with MinRK, Carol Willing, Ryan Lovett, Carl Boettiger, Rick Wagner, Sanjay Bhangar and Tarashish Mishra, as well as the experiences of the 2i2c Team in managing a lot of user images.

Essential Complexities

Currently, JupyterHub requires that there’s a jupyterhub-singleuser executable (or equivalent) inside each user’s environment it has to spawn. Without this, spawning would fail.

This executable has the following responsibilities:

1. Endpoint Security: Perform OAuth with JupyterHub, so only authorized users can access the user’s server
2. Activity Reporting: Continually report back to JupyterHub if the user has performed any ‘activity’, primarily for helping with culling inactive servers
3. Supervise and Proxy: (With jupyter-server-proxy) Start and proxy arbitrary web services (while often ignoring their endpoint security) to support alternate UIs (such as RStudio or VSCode)

Existing Implementations

There are two real implementations of this:

1. jupyterhub-singleuser the canonical implementation, relying on Tornado and Python. jupyter-server-proxy needs to be separately installed for (3). This is the most commonly used implementation by multiple orders of magnitude
2. jhsingle_native_proxy which relies on jupyterhub-singleuser but also combines with it jupyter-server-proxy and nbgitpuller

Accidental Complexities

Multiple levels of Python Environments

Wether separate or not, you end up with at least 3 kinds of Python environments:

1. The JupyterHub control plane environment, which runs just the JupyterHub server.
2. The jupyterhub-singleuser environment, which needs to have both jupyterhub-singleuser but also requires a compatible jupyter_server implementation, regardless of wether a Jupyter Server based application (primarily JupyterLab or Notebook) is being used at all
3. Individual Python environments for any kernels that may be used by the user. This can be many different ones, if say nb_conda_kernels is used.

Singe jupyterhub-singleuser requires a jupyter_server, those two are conflated. This unfortunately means that the environment that is the requirement for JupyterHub to work correctly also ends up being the environment where you need:

1. A Jupyter Server (often jupyterlab-server) and all its dependencies (50 unique ones at time of writing)
2. A valid Python version that is supported by jupyter-server (Python 3.9 at time of writing).
3. The same environment must be used for all other jupyter-server extensions you may want to install, as well as extensions to those extensions. This includes JupyterLab extensions, as well as jupyter-server-proxy extensions

And often, (2) and (3) are conflated together as well (repo2docker often does this by default, and so do many popular Docker images and conda installation tutorials). This means the version of Python the user wants has a complex relationship with the version of Python that jupyterhub-singleuser needs.

This is all purely accidental complexity, since none of this is essential to the three duties that jupyterhub-singleuser practically fulfills. The primary cause of this accidental complexity is accidental conflation caused by using the same language (Python) for both infrastructure pieces (jupyterhub-singleuser), UI pieces (jupyterlab, jupyter-server-proxy) as well as user code pieces (numpy, pandas etc). While theoretically you could have different python environments doing all these different things, most commonly they are not. At best observed environments may have one for the infrastructure and UI pieces and one for the user code pieces, but that still is conflation.

However we design the next version of jupyterhub-singleuser it should make the default happy path to have lesser conflation of these environments.

Social perception of “First Class Citizenship”

It has been possible to robustly run RStudio and many other applications on JupyterHub for many years now (Ryan Lovett and I created jupyter-server-proxy in 2016, 9 years ago). However, it is still perceived that JupyterHub is primarily for running “Jupyter” as broadly defined, which to most people implies JupyterLab for better or worse. While this is technically not true to some extent, it is socially true. There is an additional implication that JupyterHub is primarily for Python workloads, due to the strong association of Jupyter with Python. All the R users I know use RStudio.

There is a technical reason for this too - jupyterhub-singleuser requires a jupyter-server, and most often this means you are installing JupyterLab. Which by default brings in ipykernel. So even a purely R and RStudio focused environment like Rocker has to bring in JupyterLab and with it a Python kernel.

So the default path to running non-JupyterLab applications on a JupyterHub involves installing both JupyterLab, and a Python environment. This has the social and technical consequence of making it clear that JupyterLab is the first class citizen, and everything else is something along for the ride. This causes practical issues too - the folks running Rocker recognize that RStudio support in JupyterHub is second class, and that does make them less inclined to continue supporting it if drastic changes are required.

This is also an accidental complexity - nothing about the inherent essential complexities of what jupyterhub-singleuser needs requires any of this to happen.

Building user environments requires understanding the Jupyter ecosystem

To build environments (either docker images or something else) that work well with JupyterHub requires the person building it to have a detailed understanding of:

1. The environment builder itself (Docker, conda, VM provisioning, etc)
2. The specific tool that is used to provision the user code support environment (conda, various R install mechanisms, venv etc)
3. JupyterHub specific knowledge (Which environment needs jupyterhub-singleuser, how to install Jupyter Server extensions, Kernel discovery, etc)

(1) and (2) are very broad skillsets that are widely applicable, and widely supported. (3) however, is niche and something only we, the JupyterHub community, can provide. We don’t provide it very well - this snippet in the repo2docker docs is what I could find, and I know it’s not complete nor enough. This is particularly a barrier for people who aren’t planning on using JupyterLab or Python.

Desired Outcomes

Before proposing any specific solution, I want to list the desired outcomes I have for what such a rethink would do.

1. Non JupyterLab, non Python communities are treated as first class citizens. This is a social outcome, and the most important one for improving the breadth of the JupyterHub community, to serve all our users where they are, and perhaps, the longevity of the JupyterHub project. Wether we meet this or not should be gauged by the presence of such people in our community of contributors and maintainers.
2. JupyterHub is recognized as ’not just for Jupyter(Lab)’. This is already (mostly) not technically true, but is perceived to be true for accidental complexity reasons. Let’s make it technically fully not true, and socially not true as well.
3. Building environments that work with JupyterHub become trivially tacklable without having a deeper understanding of the Jupyter ecosystem.
4. Easy transition for existing JupyterHub deployments. It should be more like the R 3.x to R 4.x transition, rather than the Python 2.x to 3.x transition. This includes, but is not limited to, compatibility with existing pre-built environments as well as not requiring duplicate maintenance of code on an ongoing basis.

Any proposed solution should be evaluated along how it will help us as a community achieve these three outcomes. This removal of accidental complexity will also make general maintenance easier, as well as expand the group of people who feel they ‘belong’ to the JupyterHub community and can hence help with maintenance.

Implementation requirements

1. IdentityProvider integration
2. Activity Tracking integration
3. A well understood API for passing authorization information on a per-request basis
4. Deployment considerations
5. Community and ongoing maintenance considerations

Needs more investigation

1. A deeper understanding of the IdentityProvider integration that JupyterHub currently does with Jupyter Server needs to be included in the outcomes.
2. We do a bunch of XSRF munging. Need to understand how much of that the proxy needs to be able to do, or if it even can

Potential Inspiration

This section lists some open source applications we can take potential inspiration from.

1. JupyterHub itself! I think our use of OAuth for endpoint security is quite nice, and I think we can learn a lot from what we do.
2. 12 Factor App Philosophy which inspired a lot of how jupyter-server-proxy does things.
3. Open OnDemand which is more focused on HPC applications and slightly different architecture. It does use modified nginx in place of jupyterhub-singleuser.
4. https://github.com/OpenIDC/mod_auth_openidc is a classic Apache2 implementation of a proxy that authenticates with OAuth2 and then also potentially spawns & supervises processes. We should not use httpd, but helpful to not reinvent concepts

Calls to action

Do you want to fund this work? Or take this work on in a sustained fashion? Post on the Jupyter Zulip and tag me (@yuvipanda)!

Endorsements

Here’s some endorsements from some people :)

MinRK: “i think that’s 100% worth doing if/when someone has the time, and appropriate to publicly describe as a goal. we would need to define a new spec for authenticating requests from the proxy.”

RyanLovett: “Just a reminder about https://github.com/jupyterhub/jupyter-server-proxy/issues/1, which seems relevant.”

Rick Wagner: “This absolutely worth scoping and determining if there is capacity to begin work.”

As many of you know, I have been doing a lot of intense therapeutic work over the last few years, with the primary goal of being more consistently emotionally regulated under a variety of circumstances. I have observed a few hard fought but helpful changes in my behavior, and wanted to explicitly write about them.

The primary audience is people who may have known me for a long time in specific capacities - the hope is that this helps provide context if they notice me behaving differently. The behavior difference maybe intentional, or unintentional! If it is in this list, it is intentional. If not, it may or may not be intentional! Talk to me about it if you would like to.

While this is an observational list rather than an aspirational list, I am human and recognize that I can not promise consistent behavior. So don’t think of these as contracts, but do think of these as helpful guidelines for interpreting my behavior if they seem different than before.

Surface and handle conflict, rather than avoid it

I have gone to great lengths to avoid conflict in the past, to the detriment of everyone who should have been involved in that conflict. Conflict exists wether I like it or not. I did not necessarily have the tools to handle conflict in a healthy fashion, so I was avoiding it no matter the cost. This meant I wasn’t gaining experience with handling conflict, so I wasn’t good at it, so I avoided it harder. A terrible negative spiral.

I have better conflict management skills now than I did before. There is always more to learn, but I no longer treat “avoid conflict” as a goal in itself. I am getting better at being able to notice existing conflict, and take intentional action - surface it so it can be dealt with, or intentionally ignore it. As I keep doing it, I get better from experience. A positive spiral!

In the past, this perhaps meant that I would only explicitly surface conflict way past due, when it has metastasized (in reality or perception). Now I am able to do it when it is small, and hence easier to handle. There has been and continues to be a backlog of ignored conflict I continue to deal with however I can.

So, if it feels like I am bringing up more conflict than before, it is not because I hate you. It is because I am no longer ignoring it.

Recognize that friends and colleagues / collaborators are distinct relationships

Friend and colleague as distinct but potentially overlapping relationships. People will have different expectations of me, and I of them based on the kind of relationship I have with them. And this is ok.

I grew up in a way where a lot of my early friendships were open source adjacent. But as my life evolved, I was treating every single open source interaction as a friendship. This was simply inaccurate - while many people I met via open source treated me as a friend, to most I was simply a collaborator or colleague.

Expectations of friendship and colleagues are indeed fundamentally different, and I got my heart broken a few times (“How could a friend treat me like that!?”) before I realized I was pushing friendship expectations on people who never thought of me as their friend. That is not fair to them nor to me. I now accept that a Collaborator / colleague (a blurry distinction in open source) relationship is valuable in itself, and can be deeply fulfilling for everyone involved even if there is not a personal friendship. These are just different kinds of relationships - not directly comparable, one is not ‘better than the other’. This does not negate the many people in my life who are both collaborators and friends. Just that this distinction does exist, and just because I see you as a collaborator doesn’t mean I necessarily see that relationship as “less” valuable or respectable. Just different.

Accept I will make mistakes, and don’t shy away from feeling guilt

I now think of shame as ‘something is wrong with me’ and guilt as ‘I made a mistake’. Being able to separate those different emotions has been a fundamental breakthrough for my sense of emotional regulation. Shame is to a broad extent not helpful to me, while guilt is enormously helpful. I don’t shy away from feeling guilt anymore - it simply means I made a mistake, something I can own up to, and do better. That is unlike shame, which has no recourse for moving forward from. They are both uncomfortable emotions, but guilt is useful while shame is not. Being able to observe whenever I have felt shame, and lean into the underlying emotion there has been extremely helpful.

An upshot of this is that I no longer fear guilt as an emotion, and so am more open to making mistakes. I am free from a default of the ‘freeze’ stress response! I am able to more directly feel the emotions I am feeling, instead of spend time feeling emotions about the emotions I am feeling.

This doesn’t mean I treat people with any less care or respect or thoughtfulness than before! Not an excuse to be a dick. But a pointer to not let perfectionism, the fear of uncertainty, or the intolerable physical feeling that comes with shame to prevent me from living my life.

Not Complete

This is not a complete list of course, but hopefully this is helpful for someone to read. It was helpful for me to write!

Recently, I started becoming aware that I hold opinions and desire choices outside what is considered “mainstream” in many circumstances. Some examples are:

1. Motorcycle safety
2. Circumstances under which I will be wearing an N95 mask (far more often)
3. Social Media use
4. (A few more that you already know about if you know me well enough)

When I was younger, this used to bother me. Observing my behavior, I seem to have responded in one of three ways:

1. Try to put effort towards changing what is the mainstream, so I now fit within it. Primarily this was me trying to “convert” people around me, often in irritating ways.
2. Absolutely loudly hate the mainstream, and make it out so that being seen as outside the mainstream (in exactly the way I am) is a morally superior position
3. Quietly deny my opinion or desire to myself, and try to convince myself to change so I can be within the mainstream

I see all these as unconscious coping mechanisms to ward off my own discomfort and fears about being vulnerable. They all suck, cost a lot of energy, and don’t actually seem to comfort me.

There is a secret 4th thing that I have sometimes done unconsciously, and now I want to consciously do more often:

1. Evaluate if my opinion or desire is fully true to myself. If so, clearly accept and communicate it, without any reference to other people’s opinions or choices. If not, change my opinion or desire something else.

This is not me ignoring other people’s opinions - but forcing myself to fully own my opinion, to have enough clarity so I can communicate it appropriately, and give myself permission to change in light of new information. It feels vulnerable yet immensely freeing.

I will practice this, so I will continually try, fail and improve in a loop. I am writing and publishing this primarily as a reminder to myself.

Jeff Nippard (the science communicator you watch if you want to spend your evening reading scientific papers and going to the gym) wrote a beautiful post recently about handling harassment and bullying at Gyms. One part that really stuck out to me was:

Also, every gym has at least one “big [person]”. Almost every big [person] I know is a good person. I’m sure big [person] already all know this, but people find you intimidating. Even a simple smile toward new members can do a LOT.

While I’m not a ‘big guy at the gym’ at the gym, often I do find myself being a ‘big guy at the gym’ in many of the open source communities I have been a part of. I have been writing code for 23 years, been part of various open source communities, and have a lot of power in many of them. I recently realized that I am one of the original creators of about 1/4th of the various projects in the jupyterhub GitHub org, for example!

This means I have a lot of power to make people feel welcomed. Open Source communities can be an intimidating space, especially if you aren’t a cis white man. I’m not, and I also have a lot of power! A nice and welcoming word from me here and there really helps. I know because others have done this for me in the past (Kyle in particular for example). I have historically tried to do this as well, but didn’t have a proper term for it other than ‘be nice’. Now I can say ‘be the “big person at the gym” for your community’.

There was an opportunity to practice this yesterday, and I think I did well! It was fulfilling to me, and took me only a few minutes.

You are probably a “big person at the gym” for some community you are a part of too, wether you realize it or not. If you aren’t a cis white man, it’s more likely you don’t realize it. Think about it, feel proud of yourself, and smile at the newbies.

If I want to share a YouTube video with someone while on my phone, here is what I need to do:

1. Click the ‘Share’ button on the YouTube app
2. Click ‘copy link’
3. Paste it into a textbox where I can edit the URL (so if I am using iMessages on the iphone, I have to use a text editor app in the middle)
4. Delete the ?si=ajflsdkjf at the end of the URL added by Google so it can establish a relationship between me and whoever I am sharing it to (THIS IS CREEPY AS FUCK)
5. Share it with the other person.

I’m tired of doing this, but I also feel a little angry each time I do it. Millions of dollars of engineering time have gone into building the infrastructure and UX for this si stuff, and it also exists as a response to regulatory crackdown on creepy tracking.

So every time I do it, the emotion I feel is ’this technology hates me, it is fighting me, and I have to fight back’. I don’t think that’s quite right, but that’s how I feel, and I do not like it.

I do pay for YouTube premium, because the ability to background play videos on mobile is gated behind that. So I can’t really just use a browser, where I can use extensions to do this automatically for me.

I just want to not fight the ’tools’ I am trying to use every day. I want them to serve me. Is that too much to ask for?

Julia pre-compiles packages on first load, allowing them to deeply optimize the generated code for the particular CPU architecture the user is running on for maximum performance. However, it does take some time, so there’s a startup latency penalty here. If you make docker images with precompiled Julia packages, you can pay this pre-compilation penalty at image build time, rather than at image startup time. Much better for your users!

If you look at the tutorial for pre-compilation, it may sound like all you need to do is call Pkg.precompile. And if you actually build and test your docker image, it will work fine - your precompiled libraries are loaded, and startup is fast. Yay!

But then you push your built image to a registry, and someone else pulls it and runs it - and the pre-compilation has no effect! Packages are pre-compiled again on first run, and sometimes this may take multiple minutes, causing weird to debug timeouts. But then you can’t reproduce it, because it runs fine on your computer! What’s going on?

What is going on is an abstraction leak. We think of computers as ‘computers’, and often it’s easy to forget that there’s just a wide wide variety of them. After a few decades of Intel’s (or should I say AMD’s) amd64 architecture hegemony, M1 Macs (and Raspberry Pi) have at least forced us to consider that ARM64 exists as well. But those are still just broad classes of architectures, and each generation of CPUs within them also have differences, with specific instructions available in some that aren’t in others.

So turns out Julia’s pre-compilation is CPU dependent (or more specifically, ISA dependent). So if I build my docker image on one type of CPU, the pre-compiled files will only be used when running on the same type of CPU!

The limits of ‘works on my machine’ :)

This manifested as a ‘Pluto notebook is not starting when run on JupyterHub’ issue. It was fun debugging this, eventually leading me to discover this bit about Julia pre-compilation. We eventually landed a fix that cross compiles to a number of different ISAs, producing a ‘fat binary pre-compilation’ that starts fast on a number of different CPUs!

I’m also personally happy with the amount of time this took - I’m trying to learn to wean myself off high stress ‘gotta fix it now’ open source maintenance, and a slower paced but more consistent approach.

Anyway, if you’re building Docker images with Julia in them, and want to pre-compile any packages you install in the image (as you should), I recommend the following bash snippet before you call Pkg.precompile

if [ "$(uname -m)" == "x86_64" ]; then
    # See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L24
    # for an explanation of these options
    export JULIA_CPU_TARGET="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"
elif [ "$(uname -m)" == "aarch64" ]; then
    # See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L54
    # for an explanation of these options
    export JULIA_CPU_TARGET="generic;cortex-a57;thunderx2t99;carmel"
fi

This is what the official Julia binary is built with (and you should be using that, not your distro provided binary). It may need updating as new ISAs come out, but otherwise should help.

Thanks to mathbunnyru and benz0li for helping with this.

P.S. Did you know that Python also does pre-compilation, and that can also have a massive effect on python startup time? I did a similar debugging session on that a while back.

Why write this?

This August / September, I had a lot of life changes in one go:

1. Moving from an apartment i disliked to one I absolutely love, and maybe even feel at home in, for the first time in my life
2. Officially switch jobs, becoming an employee of a non-profit I had co-founded
3. Catch COVID for the first time and recover from it, with appropriate MH breakdowns from the isolation
4. Get on Guanfacine as a non-stimulant medication for ADHD

While I can not fully separate the effects of these changes, the overall effect has been very good for me. The ADHD meds form an important and specific component, and I think I am primarily writing this to understand what that is. Many years ago, I had started on bipolar 2 meds and anti anxiety meds at the same time, and attributed some positive changes to the bipolar 2 meds that were not true - it was almost exclusively the anti anxiety meds. This contributed to me living with a misdiagnosis for many years, and taking meds I just didn’t need. So I don’t want to unconditionally say all the good positives in my life are from this, but recognize that some are, especially this time we did not mix it with another pharmacological intervention. Writing it down now should also allow me to come back to it a year or two later to see how that has evolved

Dosage & Side Effects

I currently take 2mg a day in the morning, after starting at 1mg. No plans to increase dose further. I had some dry mouth for a week or so when starting, but it has mostly subsided.

I had been on Ritalin for a few months when I was first diagnosed (happened when I was 20). It was an important marker in my life (my first thought when it kicked in was “Wow, is this how normal people feel?”). I did not enjoy the “crash” in the evening, but now i suspect increased dosage or an extended release would have helped fix that. But I was still in Chennai at that time, and due to various socioeconomic factors + how hard it was to access, I gave up after 2 months. While I got back on other psych meds later, due to various other mental health comorbidities, I’ve been wary of trying stimulants for myself. I’m extremely happy that there are now non-stimulant options for ADHD, and one of them is working for me.

Productive capacity

The first, and most noticeable difference is that doing “work” (broadly defined as ’thing that pays my salary’) is easier. Specifically,

1. I can focus on a single problem for a longer stretch before I lose energy and hence focus. My work sessions now seem to be closer to an hour before a break rather than 20minutes before a break. This also means the total amount of time I can bring to bear on a problem per day has increased, as much as by 50%.
2. I seem to have more control over what I am actually interested in. I know I do great work when I focus on something, but I never felt fully in control over what things I could choose to put my attention on. We have to evaluate this more long term, but so far, this atrophied muscle seems to be slowly growing.
3. I am able to actually plan my day and do the things I have planned, and say ’not now’ to other things I may want to do during the middle of the day. I have never managed to stick with any kind of personal planning system. What I had planned and what I would actually do would often not match up, causing some amount of emotional pain (“Why am I so incompetent?” usually). After a while, I’ll give up said system because the pain is just too much, instead chasing the ephemeral joy of just doing things.

So my overall capacity has increased (1), and so has my ability to apply it in a particular direction. This is awesome. It’s now up to me to work on building systems that allow using these gains, building organizational muscle that was just not possible before (3). I’m excited to build this muscle, re-trying things that I had tried before and failed at. One specific hope I have is that it’ll help me go from someone who gets something to 80% completion to someone who gets things to 100% completion more times.

A side-note here here about my ADHD Tax, which has been primarily paid as shame and emotional turmoil (ask anyone who has been around me whenever I have had to apply for a Visa), although of course there has been a financial and opportunity cost. This has gotten so much better. I just got a ton of pending paperwork finished, without the usual attached emotional turmoil. For the first time in my life, I think I will be able to get on top of this and stay on top of it.

Emotional capacity

Between the stimulus and response, there is a space. And in that space lies our freedom and power to choose our responses.

This quote from Victor Frankl has always stuck with me, and I think Guanfacine has just increased the amount of this space I have to do this.

I have been in therapy for a long time, and will probably continue to be forever. I have learnt many skills I can bring to bear, particularly at times of extreme (and often disproportionate) emotional distress (something that seems to happen far too frequently for my taste). However, often when you are emotionally distressed, you just don’t have the capacity to use those tools! The best you can do is to avoid causing harm, to yourself or others, and retreat for your body to recover.

Guanfacine has definitely made this situation better. I have more capacity now to actually use the tools I have learnt, and iterate on them. Insight is often useless without action, and Guanfacine has given me far more scope for action than I had before. This isn’t something that ‘heals’ everything (no such thing, unfortunately) - but it has vastly increased my ability to ‘do’ rather than just ‘know’ or ‘feel’. Very grateful to have it.

What now?

My overall goal has been to find ways to heal and lead a less miserable, more joyful life than I have in the past. I understand this is probably a years, decades-long process, and I’m much closer to the start than to the middle Guanfacine feels like a significant boost along the way here. I’ll check back-in in a year and see how the things in this post have aged :)

If you think you may have ADHD, consider getting evaluated and trying some medication. This may be a fucked up process that you may not be able to afford for economic, social or even just ADHD reasons (may need a lot of follow through ugh). But, I think it is absolutely worth it, so if it is available to you and you have been putting it off, consider this a sign to re-prioritize that.

You can reach out to me by emailing yuvipanda@gmail.com if you wanna say hi.

Trying to start these back up, as a list of the kind of ‘work’ I have done the last few weeks. Let’s see if this works!

Last 2 weeks

1. Spent a good chunk of time investigating and fixing issues with AzureAD auth for the UToronto hub. “Fixing” it meant moving off AzureAD, so we only have CILogon & GitHub in the 2i2c infrastructure as authentication providers! This is now explicitly mentioned in our docs. An incident report still needs to be written.
2. Upgraded Azure AKS for UToronto, leading to a pretty gnarly outage. Incident report needs writing. issue.
3. Finished setting up the ClimateMatch hub. Part of this involved adding terraform support for proper, single-hub nodepools in GCP.
4. With help from @sean-morris, all Cloudbank Hubs now use a data8 style image. This has reduced the number of non-staging hubs using the old ‘default’ 2i2c image to 5! When unspecified, the default image in use now is jupyter/scipy-notebook. These communities should be contacted and a different image be used for them, so we do not have to maintain this ‘default’ image anymore.
5. With the goal of removing special cases from the 2i2c infrastructure, I am working on removing the manually maintained NFS servers from our infrastructure. I had thought only the 2i2c shared cluster had it, turns out the cloudbank one does too! The 2i2c shared cluster has been migrated off, the cloudbank one will be soon. With that, home directories are unified for all clusters, and no longer require specialized knowledge of filesystems, etc to manage.
6. Did a bunch of cleanup work on our terraform code.
   • Remove unused AzureFile SMB support, as we use NFS everywhere. Yay consistency. Pull
   • Fix terraform linting errors and fix the job that was doing the lint to actually do the lint. Pull.
   • Use optional fields in our terraform variables, for cleaner .tfvars files. Pull

Upcoming week

• Help with stabilizing quarterly goal for next quarter for 2i2c.
• Finish two incident reports for UToronto so we can close that out:
  • https://github.com/2i2c-org/infrastructure/issues/2628
  • https://github.com/2i2c-org/infrastructure/issues/2594
• See this PR through to completion: https://github.com/2i2c-org/infrastructure/pull/2406. Mostly documentation.
• See this PR through to completion: https://github.com/2i2c-org/infrastructure/pull/2621. Mostly upstream documentation, started in https://github.com/jupyterhub/grafana-dashboards/pull/84.
• Finish moving cloudbank hubs to filestore, decomission old NFS VMs. https://github.com/2i2c-org/infrastructure/issues/1105

I’ve had a bit of a slump the last few months, working through what feels like deep mental fog for totally unclear reasons. The lack of clarity on causation is one of the most frustrating parts of this, and one of the most familiar too - this isn’t the first time I’ve felt like this, and I know (rationally at least, if not emotionally) it won’t be the last. Probably a combination of intensely avoiding making some decision that seems too big to be made by me without actually knowing what it is and fun brain chemicals stuff. Who knows, I’ve given up on trying to figure out causality for these things for the most part.

One of the effects has been a distinct lack of drive, at least to the extent I am used to. Another is that it is difficult to remember being in a different emotional state in the past, and difficult to imagine being in a different emotional state in the future. Time dissolves, and I believe how I feel now is how I have always felt, and how I will always feel.

Hence, am trying to make external records any time I feel or encounter something that seems to give me a good boost, and a sense of ‘wanting to do it again’. That in a loop feels like drive to me - A feeling induced in you by external stimuli, and then you wanna find things to do to feel that again.

I felt some of that a few days ago! I built something, put out a quick message about what it is that I had built on a slack. Someone responded with this message:

And that felt really good! I could see how responses like this have driven most of my software development projects, for good and ill. It has been a driver in picking which communities I can be a part of, what I find interesting, and what feels like a slog. I definitely find communities that are at intersections of various fields, and I try to find them early - so things I could do that would feel ’normal’ in one of those fields feels mind blowing to the other. I joked to a friend that my ‘anxiety response’ is to start (or get deeply involved in) a new open source projects, but the more I think about it the more it seems to be not a joke at all.

Not unique to myself in any way or form I would assume. But feels great to know this is true for me, and I have come to value explanations for my emotions quite a bit of late :)

Anyway, great to remember that I can feel this!

I walked into a local hardware store, with the following thought in my mind:

I need a 5mm metric hex key

And I wandered around aimlessly as I do in stores, almost bought some double sided tape for a different project (decided against it), and eventually made it to the place with all the sockets, hex keys, screwdrivers, etc.

Words that previously meant nothing to me (Torx, SAE, metric) made sense. Felt a little like the very first time I felt power learning to write code (the strongest memory is that of changing color of a button in VB6 with one line of code). What was noise before had meaning now.

I got myself a little folding metric key set with hex keys going from 1mm to 6mm - I knew I had some new windscreen clip-ons that needed a 4mm key, and am sure I’ll find a 6mm somewhere.

I pick up something that costs 13$, and looks ok. I look at a different brand, and decide I like that better because of the way it folds. “Hey, this means when I try to turn it, I’ll get better leverage!”. I actually know what that sentence means, and I choose one brand over another based on actual knowledge rather than a guess. If my programming experience is anything to go by, I’m completely wrong in ways I won’t even understand for a long time.

But I buy it, leave the store, and try using it for intended purpose on my motorcycle - to remove the bolts holding my bar end weights in. Earlier that week, I had stripped these by attempting to use a friend’s impact driver with a torx bit (it sort of fit, and I had no mental model about what I was doing!). The head stripped, and the thread stripped, and I took it to Performance Motorcycles in Berkeley who fixed it up for me (although the bolt itself did break and they had to put a new one in).

I had since beat myself up about not knowing enough about hardware to not strip the head, been comforted by a friend, realized that I had set weird standards for myself where somehow I must already know all of this having never put in any effort to learn, and then being delighted that ‘oh wait this is just another skillset I can pick up’. It will probably take years, but that’s ok!

Earlier that day, I had went back to the mechanic and asked him exactly what screw he had used for the bolt. He told me it was a 5mm hex, and I should use an exact one or I will strip.

I extend the 5mm key, and fit it in. It seems to be a perfect fit! I try twisting left, repeating internally to myself “Lefty loosy”. It doesn’t budge, and I’m slightly anxious. But I trust that it is not super likely to strip at this point - I’m not a very strong person, and this I fully believed was the correct size. So I put more energy into it.

AND THE BOLT TURNS! I FEEL ELATED! AMAZING JOY! It is a bit like seeing your code run for the first time, AND IT FEELS AWESOME!

So excited for this future :)