My UploadWizard work has just been deployed to Wikimedia Commons and my name is listed in the on site credits. Special thanks to Mark Traceur for code review!
This feels very, very nice :) Now to sneak more code into other extensions…
Talking to God is an interesting piece about apparent ‘dangers’ of knowledge and the necessary side effects of evolution. Thought provoking.
When running Redis in a shared cluster/hosting environment (such as Wikimedia Tool Labs, on which I’ve been having fun doing a lot of work on), you would want to try to provide at least some guarantee of isolation for your keys from everyone else’s keys. Since Redis doesn’t do ACLs, this is problematic.
This can be solved in a couple of ways
This is simple enough to do – each user runs their own Redis instance, and has full access to it. Security is handled by setting a secret password, and running redis-server as the user in question. Boom, secure!
This doesn’t really scale with a large number of users, because they each have lesser memory to work with now. Also having users who just want to run their tools have to deal with making sure their Redis instance is up and running fine isn’t really good. Having the sysadmins be responsible for users’ Redis instance is… not going to work :) This would also require all the redis instances to run on one box and/or have a separate cluster just for them, which isn’t good either.
Not happening, because I’m not good enough to do that yet :P But more realistically, it won’t ever happen, since this will probably add a lot of overhead for what is arguably an edge case.
Such a server would simply authenticate incoming requests via some mechanism (Keystone perhaps), and then enforce ACLs. It will have to speak the exact same protocol as Redis, since users should be able to use any library that connects to Redis. This isn’t too hard – just replace the password functionality of Redis protocol to take in both a username and password (or token, or some other method of auth).
This also I discarded – it will still affect performance, which will now be limited by how fast this server runs, and that is definitely not faster than Redis. And it will also be hard to maintain, since I’ll have to completely mimic Redis’ protocol and make sure it is kept up to date. Debugging protocol issues with random client libraries is not my idea of fun. Another major disadvantage is that I would now be writing auth code, and I don’t think handrolling auth code is a good idea, ever.
This is what we finally settled on :D It sounds horrible by the title, but I think it is Good EnoughTM.
Since Redis is a key value store at heart, you can do anything once you know the key. So, if an ‘attacker’ doesn’t know the key, there isn’t much they can do. So it can be considered SecureEnough for our purposes if we can make it so that other users can not find out or guess your keys.
We essentially did so with the following:
(1) prevents someone from just listing all keys to find something interesting. (2) prevents people from brute-forcing or guessing keys. Since all code run on Tool Labs must be open source, guessing keys is super easy. By having a ‘secret’ prefix, having the actual keys is useless. This also prevents accidental key overwrites from different tools using a common key name.
Disabling commands is easy to do by using Redis’ RENAME COMMAND config feature. I added support for RENAME COMMAND to Wikimedia’s Redis puppet module, and then it was simple enough to configure a specific instance to disable ’list keys’ type commands. That’s the following commands:
CONFIGFLUSHALLFLUSHDBKEYSSHUTDOWNSLAVEOFCLIENTRANDOMKEYDEBUG.After going through the list of Redis commands, I am guessing this is going to be GoodEnough to prevent key listing. (Note: if there’s more that I’m missing, please, please let me know).
We also tell people to use a secure prefix that’s at least 64bytes long, saved in a file that is only user readable. Generating that is as simple as:
openssl rand -base64 64
That should be long enough to be hard to brute force, even with Redis being as fast as it is.
The major problem with this is, of course – the fact that humans are involved :) I’ve heard “I do not care about my keys, do not need security” a fair amount of times already. The fact that the prefix generation is optional means that there will be people who do not use prefixes, and it will work for them for a (probably) long time – until it doesn’t, and they have no idea why. This is personally acceptable to me, since they have been made aware of the risks beforehand.
This has now been deployed on toollabs for a month or so, and I’ve a couple of fun tools already written using it (and other people too). We had a patched memcached server we had that we’ll kill in a few weeks, so people who used memcached before are also migrating to redis. And I was able to do all this without even having root! This is mostly thanks to the fact that we try to keep all our configuration in puppet (Wikimedia’s Puppet repository) – for both our production cluster and for everything else. So I could re-use our production redis module, make changes to it, and build the new solution – all while being vetted by ‘proper’ ops people (whom I dearly love and respect). Building infrastructure in such a collaborative manner is a lot of fun, and I think I’m hooked. It’s fun!
Ever since the death of Windows Live Writer, I’ve never managed to find a nice blog editor that I really liked and also worked fine. I had a nice Vim plugin that posted to wordpress for a while, but that seems to have stopped working. The WordPress interface itself is mostly fine, but still a fair bit ‘meh’.
Platen is a markdown based editor that’s a Chrome Web Store app that looks cool so far. Let me see if it actually works.
Comedy Central has been banned in India for the next ten days. Reasons?
Whereas, the programme showed a man performing an act of stand_up comedy and mouthing vulgar words accompanied by obscene and suggestive gesture and gyration. During his performance, the man was shown uttering dialogues denigrating women, indecently and crudely referred to sex organs of men-and women and the sing-song rendition by the man sought to pornographically describe .the male lust, depicting women as a commodify of sex. The portrayal appeared to deprave, corrupt and injure the public morality or morals
By that logic, I suppose 90% of all Indian Movies (and especially songs) should be banned too.
“Science progresses one funeral at a time” – Max Plank. Not just science….
Potential trigger warning
I keep trying to tell myself “People are not evil. They are just doing the best they can with what they have”. But then I’ve to read shit like this, and I’m not so sure anymore.
Growing up to lose respect for people you considered heros is a horrible thing. Friends don’t let friends fall prey to emotional blackmail.
Screenshot of Autosuggest working in Tamil on my variant of the Android keyboard. I’ve been working on this on and off for the last few months (this is also my final year project in college). I’ve ported all of the jQuery.IME languages to work on the keyboard natively.
Now experimenting with Autosuggest. It only needs an appropriate dictionary now and we’ll be good to go. Currently I’m getting Autosuggest working only on Tamil, but the plan is to get it to work for all Indian languages. The method I am using (convert everything back to latin chars, then do autocorrect / autosuggest) means it should be trivial to extend to any transliteration based input method.
Wheeee! This is very much a simple ‘celebratory’ post. Will blog actual details soon.
Not sure if you’ve noticed, but over the last few days Wikipedia has a new Favicon – featuring (according to the commit message), “Multi-resolution, new render, rounded corners, very confused and angry graphic designer” :)
New Favicon:
Old Favicon:
Looks much nicer when not on a light background.
Potential trigger warning
Like if you ask three separate groups of Canadian households how much they’ll pay to save two thousand, twenty thousand, or two hundred thousand birds from dying in oil ponds, the three groups will respectively state that they’re willing to pay seventy-eight, eighty-eight, and eighty dollars. No difference, in other words. It’s called scope insensitivity. Your brain imagines a single bird struggling in an oil pond, and that image creates some amount of emotion that determines your willingness to pay. But no one can visualize even two thousand of anything, so the quantity just gets thrown straight out the window.
– HPMOR on Scope insensitivity
Leading causes of preventable death worldwide
6 million children dying of hunger every year
[4 kids dead every minute in India just from preventable diseases
]8
As brion mentioned on IRC a while ago, us humans aren’t really doing that good, are we?
I feel helpless and crushed. I am helpless and crushed. But not as much as the 20 kids who died in this country while I was writing this post.