Yuvi Panda

JupyterHub | MyBinder | Kubernetes | Open Culture

liberal software

I ran into this thought provoking though when randomly attempting to relax this weekend. There’s a summary at LWN if you do not want to watch the talk - but as the lwn summarizer admits, the video definitely conveys things that are hard to capture on text.

The core takeaway for me is to think about:

what is the future of free and open-source software? The answer was: it has no future.

This seems somehow connected to ‘democratizing programming’, which I had earlier given a talk about. Somehow, it feels like there needs to be an update / rebirth of the GNU Freedoms for the world we live in.

my first protest

I went to the protests at SFO last weekend. It was the first real set of protests I’ve been to. I write this to attempt to capture a sliver of what I felt that day.

I was there for about 10h on day 1, and came home exhausted. I went back on Day 2, and this time stayed for much shorter period of time (~4h?) before heading back home.

To everyone who was at the protest even if it does not directly affect you yet - thank you!

To the wonderful amazing women of color who were leading the protest - thank you. You are an inspiration.

To the brass band and the troupe of drummers who showed up - thank you!

To the people I already know from other contexts who I ran into at the protest - thank you!

To those who were protesting for the first time in their lives (as I was) - thank you!

To the fucking ACLU - thank you! I’ll send you all the money I can :)

To the people who provided infrastructure (food, water, snacks, first aid, printers, internet, etc) - thank you!

To the journalists who covered this - thank you.

I woke up that saturday feeling very depressed, angry, and helpless. By sunday night, I was only depressed and angry - but not helpless. The number and variety of people at the protests was very heartwarming, and made me feel distinctly not alone. Before going I was not sure what going to a protest accomplishes. I still do not know - but I know it gave me hope and restored my sense of agency.

I promise I’ll continue doing all I can, even and especially when it is about things that do not affect me yet. I hope you do too.

What hurts the victim most is not the cruelty of the oppressor but the silence of the bystander

- from a sign at the protest

making twitter useful again

I’ve unfollowed everyone I follow on Twitter, and am slowly starting back up from scratch. I’m only going to follow people who are:

  1. Underrepresented people in Tech (as broadly construed)
  2. Journalists

And that’s it. I’ll follow back friends I’m not otherwise in contact with as well, but might take a while.

I’ve had this for a few days and am already enjoying using it far more than I did before. A lot of the people I’m following now I did not know before, and the vibe is totally different. I’m able to understand and appreciate things I was not able to before. I know this is a bubble, but it is certainly a different bubble than the one I was in before. Bubbles also intersect - I’ve interactions with other types of people elsewhere, just not on Twitter.

If I unfollowed you, don’t take it personally! DM me for a phone number you can use for Signal / WhatsApp / Telegram - that’s how 90% of my social activities seem to happen these days anyway.

h/t to Jorm for the idea of targetted following!

Update: I’ve also setup LeechBlock to only allow me 5mins of Twitter every 2h, and I don’t have the twitter app on my phone. I wish there was a leechblock type thing for my Phone tho.

UCI Conflicts Class: Notes from Week 1 – Part I

I’ve started taking a Coursera class on Conflicts from UCI.

Just dumping notes from me going through the video lectures here. I’ve a bunch more videos to go through for Week 1, but dumping what I have for now.

Types of Conflict:

  1. Constructive
  2. Destructive
    1. Caused by lack of flexibility, getting interpersonal things get in the way and the sureness that ones way is the right way
    2. Leads to stiffling of anything new, resulting in ‘this is the way things have always been’ being used as an actual legitimate reason for things

Sources of Conflict

  1. Economic Conflict. Resources are scarce, people tend to hoard them. Individuals stand by and watch as other members look for resources and can not find them, without helping. Dysfunctional.
  2. Value Conflict. Ideals / Principles / Preferences. Very difficult to sort out. Some values you hold very dear might be opposite to what the environment supports. You bring your own definition of values, and expect others to be compatible with it, conflict ensues. Individuals expect everyone to behave / believe / follow same philosophy they do.
  3. Power & Control. Power some people decide who they are. Titles are important. Individual takes power in always being first to speak, forcing their decision on rest of their team. Nobody’s appointed someone to a position, and they decide they are the one who takes the lead role, even though they have not that much support from elsewhere.
  4. Interpersonal conflict. Just between two individuals who can just not get along or not have civil interactions. They might’ve been friends at some point even, but something happens and splits them off. This can also be very dysfunctional for the team they work in.

Levels of Conflict 5. Role conflict – without clear definition who plays what roles (leader, decision maker, etc). Defining a team requires outlining responsibilities and what role they play. (This can probably lead to 3, 2, 1, 4, if not handled properly) 6. Intergroup. Inter group can be same as 1-5 but as groups, but can also be about respect. Example: Sales and Marketing fighting over who is the reason their product sucks. Groups can be silo’d can can think they are only responsible for their own success / failure, without thinking of cross-team stuff. 7. Multiparty. In M&A, the ‘surrendering party’ will have individuals who will be emotional and not like that this is happening. This is a lot of (5, 3, 2, 1) – freefall for everyone! 8. International. Diverse work force, cross cultural. Far more complex

Case study: Overtime

Manager asks you to work overtime, not get paid, cites projects being overtime etc.

Multiple options – such as: 1. suck up to him. 2. ask him to fuck off. 3. report him to his own boss. 4. Say you understand his perspective, but do not want to go against company policy.

Now, the course suggests you pick (4), which is an interesting choice – I would’ve picked (2), but that is not going to decrease teh amount of conflict, only increase it. (4) is bullshyt (in the anathema sense of vague euphemism that doesn’t actually say what the person means), but is something that leads to the least amount of conflict. However, I don’t know if that really is the best way to handle this shitty situation – since that’s just going to lead to bullshit all around. It does, though, AGF on part of the manager – which depending on what you think, might or might not be a good idea. Hmmm..

Bullshit vs Conflict? Maybe you have to have one or the other. OSS projects that consider (4) to be just bullshit do indeed have more amounts of conflict.

Of course, the real solution is to be not in such a place at all, and make the environment be that way through some form or way. So maybe (4) is an acceptable answer…

Very fascinating!

Case study: Dominant team member

Putting down everyone else, making everything not go well.

  1. Why is this individual behaving this way?
  2. What do you need to do to bring the team back together in one cohesive unit?

Has even more what-I-would-call-bullshit suggestions. “Setup roles for meetings, have a discussion about norms”, etc. However, they all could possibly work, which isn’t true for what I would’ve done – which is to PM the person and talk to them.

I guess there is a difference between ‘doing the right thing as you think it is’ vs ‘doing the thing that has highest probability of getting you to objective’. Now, with corporations said objective is easy to define – make more money, I guess – while for movements it it is much harder…

Case study: Convo with coworker

You had a conversation with coworker, and didn’t feel it went well. What do you do?

  1. Explore what your feelings are, and what it is that you specifically feel didn’t go right. When you do this, you clear away emotions that could possibly be clouding ways for solutions. Focus only on the facts.
  2. Go to actual peer, be open to receiving their position / feedback. LISTEN TO UNDERSTAND BEFORE BEING UNDERSTOOD.

Interview: Brent Rasmussen

Everyone assumed they were responsible for some part of the website, and everyone had different ideas for what it is. Need to explicitly figure out who is responsible for what and who is in a ‘consultive’ role.

Positive conflict can raise the bar, for things that other people did not think are possible.

For destructive conflict, figure out when is the right time to discuss it.

Side Project: UA Emoji Firefox Extension

Note: I’m trying to spend time explicitly writing random side projects that are not related to what I’m actively working on as my main project in some form.

A random thread started by Ironholds on a random mailing list I was wearily catching up on contained a joke from bearloga about malformed User Agents. This prompted me to write UAuliver (source), a Firefox extension that randomizes your user agent to be a random string of emoji. This breaks a surprisingly large number of software, I’m told! (GMail & Gerrit being the ones I explicitly remember)

Things I learnt from writing this:

  1. Writing Addons for Firefox is far easier to get started with than they were the last time I looked. Despite the confusing naming (Jetpack API == SDK != WordPress’ Jetpack API != Addons != Plugins != WebExtension), the documentation and tooling were nice enough that I could finish all of this in a few hours!
  2. I can still write syntactically correct Javascript! \o/
  3. Generating a ‘string of emoji’ is easier/harder than you would think, depending on how you would like to define ’emoji’. The fact that Unicode deals in blocks that at least in this case aren’t too split up made this quite easy (I used the list on Wikipedia to generate them). JS’s String.fromCodePoint can also be used to detect if the codepoint you just generated randomly is actually allocated.
  4. I don’t actually know how HTTP headers deal with encoding and unicode. This is something I need to actually look up. Perhaps a re-read of the HTTP RfC is in order!

It was a fun exercise, and I might write more Firefox extensions in the future!

Paper notes: ‘The impact of syntax colouring on program comprehension’

I’ve recently started reading more academic papers, and thought it’d be useful to write notes about them and publish them as I go along! This one is for The impact of syntax colouring on program comprehension

  • I was amazed at the amount of prior research it is citing. Why have I not been reading these for the last 10 years of my life?
  • Apparently it is ok to report findings with a sample size of 10 people. I do not know how I feel about this.
  • The fact that there’s a large amount of thought put into the design of the experiment is quite nice, and surprisingly different from environments I’ve worked in the past where product managers designed ‘experiments’
  • To avoid datatype-related confusion, a uniform variable naming scheme was adopted in the tasks. For example, integers were named x, y, etc. and lists were named list1, list2, etc.. As someone pretty used to Python, I would have found this annoying – but I’m curious what the effect of identifier names is in program comprehension. It also reminded me I haven’t written any code in a stronger typed language in a while (I don’t think Java counts)
  • They used Solarized Color Scheme, which has a lot of fans although I’ve never been one.
  • Lots of self-reporting for ‘programming proficiency’. This is the ‘we give up!’ answer to measuring programming proficiency, I guess :)
  • We gathered data from 10 graduate computer science students at the University of Cambridge. This too seems fairly common, but I’ve no idea if such an un-diverse group of student group being studied affects the results at all?
  • They also discarded data from 3 of the students because they wore glasses and their eye-tracking hardware could not really deal with that. So this entire paper is from data from 7 students doing one particular course from one particular university.
  • We use the Shapiro-Wilk test to establish normality. We use the Wilcoxon signed rank test (WSRT) for paired nonparametric comparisons. I know some of these words!
  • As the data was not normally distributed, a 2-way ANNOVA could not be used to investigate the interaction of experience with highlighting on task times I know most of the words, but still can not make sense of this sentence.
  • Currently feeling very illiterate, but am sure this is just a feeling that will pass.
  • . The median difference in task completion time was 8.4s in favour of highlighting. To my untrained brain, that does not seem that much to me.
  • The presence of syntax highlighting significantly reduces task completion time, but the magnitude of this effect decreases as programming experience increase – this is their primary conclusion, which I can totally believe. But would I have believed it if they had come to a different conclusion? Would they have published it if it had? Would they have if there was more data? I don’t fully understand / know Academia enough to know.
  • I wonder if there has been research into richer forms of syntax highlighting – not just keyword based ones, but more contextual. Perhaps based on types (autodetected?), or scope, or usage frequency, or source, or whatever.

Overall, I enjoyed reading it – good paper! Thought provoking in some forms, but could’ve aimed higher, I suppose. I hope they continue doing good work!

DNS servers, localhost and asynchronous code

localhost is always, right? Nope, can also be ::1 if your system only has IPV6 (apparently).

Asking a DNS server for an A record for localhost should give you back right? Nope – it varies wildly! gives me an NXDOMAIN which means it tells you straight up THIS DOMAIN DOES NOT EXIST! Which is true, since localhost isn’t a domain. But if you ask the same thing of any dnsmasq server, it’ll tell you localhost is Other servers vary wildly – I found one that returned an NXDOMAIN for AAAA but for A (which is pretty wild, since NXDOMAIN makes most software treat the domain as not existing and not attempt other lookups). So localhost and DNS servers don’t mix very well.

But why is this a problem, in general? Most DNS resolution happens via gethostbyname libc call, which reads /etc/hosts properly, right? Problem there is that there is popular software that’s completely asynchronous (_cough_nginxcough) that does not use gethostbyname (since that’s synchronous) and directly queries DNS servers (asynchronously). This works perfectly well until you try to hit localhost and it tells you ‘no such thing!’.

I should probably file a bug with nginx to have them read /etc/hosts as well, and in the mean-time work around by sending to nginx rather than localhost.

How did your thursday go?

DevLog for 21-30 Dec 2015

Clearly I missed an entire week. I need to build a better system to make this easier…

Random notes.

  • Kicked out NFS from the Tool Labs proxies (with 1). Yay! This hopefully explains the lockup of tools-proxy-01 yesterday night, maybe? It’s been restarted since, and I hope to no longer have instances randomly locoking on me. Infrastructure standards of 2009, here we come! :D I’ve also removed NFS from tools-redis, and migrated them to Jessie as well.
  • Fixed up all the races in how kubernetes workers are setup with 2
  • Another instance is ‘stuck’ again. Sigh. AAAAAAAAAAAAAAAAAAAAAAAAA. Paravoid helped debug this, tracking it down to NFS client issues in the 4.2 kernel (See phab). I moved k8s nodes back to a working 3.19 kernel (after filing issue about the other 3.19 kernel package I tried that didn’t work).
  • Moved the tools proxies to 4.2 (lol?) after finding out huge ksoftirqd spikes in them. Let’s see if that improves things
  • I split up the individual components of PAWS, and have a working nbserve in there now! Exciting times. Need to fixup nbserve to use traitlets for config
  • Big Tool Labs outage (again!). Some tool accidentally sent about 12million job requests and crashed gridengine’s underlying backing store (BerkeleyDB). Reset to a clean slate after many hours (Thanks Coren!) and mostly things are back up now. I’m reading through the Berkeley DB reference manual now.
  • Persistance failed for ores’s redises again, mostly because vm_overcommit was turned off. Fixed in the core redis module so it does not happen again.

DevLog for 2015 Dec 20

Probably going to take it easy and chill. Already sent a trivial PR up though.

Also saw the old devlog mailing list and feeling happy memories. Clearly need to bring something back like that, but I don’t know / think mailing lists are the best medium. More thinking!

Ended up learning some Tornad and wrote nbserve to serve rendered notebooks and static files in a configurable way. I should refactor PAWS to have separate jupyterhub, proxy and nbserve pods tomorrow. Also need to test for path traversal attacks and whatnot. Also coroutine based programming is a lot easier than I had originally thought! woo!