Passing Headers along while Proxying with nginx

When you’re proxying requests via nginx, you might assume after reading [the documentation][1] that every header is forwarded, except for the Host and Connection headers.

You’d also be wrong.

nginx drops all headers with an underscore in them.

This is a configurable settings. You can turn it on with a simple

underscores_in_headers on

I’m still baffled as to *why* this is [a configuration parameter][2]. And why it is turned off by default. And why there is no mention of it in the proxy docs.

Thanks to kolbyjack on #nginx for helping me figure this out.

[1]: http://wiki.nginx.org/HttpProxyModule" data-mce-href= [2]: http://wiki.nginx.org/HttpCoreModule#underscores_in_headers

My First Program

Note: These are almost a decade old. Reproduced as close to memory as possible

My first memories of programming aren’t actually C. They were GW-BASIC, and later on, LOGO. I don’t remember any of my BASIC – but I do remember the look on my computer science teacher’s face in 5th grade when I managed to draw a flower with LOGO when everyone else was doing rectangles and lines. I vividly remember it – you draw an arc for a certain length, then stop, draw another arc in the reverse direction for the same length – and you have a petal. Then you just offset your next arc by a few degrees, repeat the previous thing, and you have another petal. Do this a few times such that offset-degreese * count = 360 and you have a flower. That I could just modify the variables length and offset-degree to draw flowers as big, and with as many petals as you want – it blew my mind. This was in 5th grade – and I immediately started pestering my parents to put me in a programming language class outside (I had already gone to classes for cough_MS Office_cough and DOS back then). And the SSI nearby happened to be teaching a reasonably cheap Unix/C/C++ class.

The first thing I remember still is someone telling me “you don’t need to type out clearscreen(); – clrscr() would do”. This was TurboC, and to me, at that point of time, a ‘good program’ was one that had a getch() call at the end and a clrscr() in the beginning :)

The Unix/C/C++ course – I don’t remember any Unix in it. And the C++ was disinteresting to me, because I saw no point in inheriting mammal from animal or car from vehicle. So I went about writing ‘C++’, which was basically C but with cout and cin instead of printf() and scanf() (I wouldn’t understand classes and how/why they were useful until I read Hardcore Visual Basic 5 and saw the ways classes are useful and how much it sucks that VB doesn’t have them).

I had written a program called DOSTutor then (end of 5th grade, I guess?). It had screens and screens of text that ‘taught’ you DOS commands, and then ones that made you type them in and ‘checked’ it. Initially I had written it as a long series of printf() and scanf() statements. Suddenly some bulb went on and I realized I could cut my program size by about 90% if I separated code and data, and put the data (screen text and responses) into arrays. The feeling you get when you can reduce code drastically with no reduction in functionality (+ easier to make ‘fixes’) is quite incredible :)

But nobody ever used that program, so I wouldn’t consider that my first ‘program’. I spent the next year or so messing around with random sound() and nosound() calls producing ‘music’ (including a few times when my code hit the sound() call, but crashed before the nosound() call – thus making the PC Speaker whine constantly, without any apparent way to turn it off. I used to just reset the computer then – until I learnt what exactly a DOS System Call meant. Then I wrote another small program that just called nosound()). I also remember messing around with BGI Graphics, mostly taking the example programs for graphics.h and modifying them to make somewhat-pretty figures.

My first ‘real’ program was an implementation of a variant of Book Cricket. It even had a stored High Scores file. It was my stint at a ‘programming center’ (SSI, for those who remember), and I kept it in a Network Drive that I accessed using ‘Map Network Drive’. The amount of fun that was had when I discovered rand() was quite something. It even beeped when you got out!

I lost it all in a server crash. Just do ‘Map Network Drive’ once and it was no longer there. I think I came home and cried :(

What was your first program?

DevLog: December 26, 2011 – Jan 3rd, 2012

Back in Chennai

  1. Moving yuvi.in to my prgmr server. I want wordpress for my blog, but I don’t want to run MySQL. Solution? Keep blog where it is (with the awesome webfaction people!) and use nginx to proxy requests to blog/. Seems to work, so yay!
  2. Rewrote tawp.in to use python/flask/memcached. Available on github. Replaced older node.js version
  3. Reading through The Art of Unix Programming on kindle. Amazing book so far.
  4. Fixed a few more bugs on Wikipedia App. I think I’ll keep ‘official’ Wiki related stuff off this, unless it’s big?
  5. Partied along new year’s. Yay. Not particularly ‘dev’, but still :)

Devlog: December 23, 2011

Worked the day first from Domlur bus terminus, and then from Exotel’s office (with @vijaysw).

  • Incorporated search results (rather than just suggestions) into the Wikipedia app. Was a slight bitch to get right, but after that have made a bunch of variants that the others are testing out to see which one feels good (Suggestion if exists, else search, or search alone). Took up most of my time.
  • We put out a dud as RC3 for the Wikipedia app. A missing defaults statement. Lesson learnt: Test the entire process (uninstall -> install -> run -> test) before passing off RCs
  • HyperJuice battery has arrived. Commence working out of random public places.

DevLog: December 21 2011

  1. livetrains work. Trying to move (minimal) build system to waf. Somewhat complicated, not as fluid as I’d like. Go to rake. Feel weird about starting from scratch, come back to waf. Not get anything done Another rake diversion. Final Makefile editing desperation. I give up – I need to spend atleast a day to get back on proper build tools. That day is not today.
  2. Added feature to change font size to wikipedia app. That took almost a couple of days and ended up being less than 20 lines of code. First was trying to enable pinch and zoom, but only for an iframe – proved impossible. And then a long time of headwalling about how I’m not able to manipulate the DOM inside the iframe (despite code in other places being able to do it just fine (no crossdomain issues here)), I had given up and moved elsewhere. Finally brion suggested I try inserting a into , which actually did work. So much for the browser being sane
  3. ‘Refined’ the search behavior of the wikipedia app a bit more. Also got bit by a difference between zepto (that we use) and jQuery (whose docs I have to read, since zepto by itself seems to have no docs). Had to read through the code to fix it, and managed to.
  4. Closed a *lot* of bugs. BOY THAT FEELS GREAT!

DevLog: December 11, 2011

  1. Added UI for AssessmentLog [SelectionSifter]
  2. Removed lots of dead code in SelectionSifter.
  3. Went to Bangalore Wikipedia Meetup. Was nice meeting people I knew online face to face. Talked about yenWikipedia, and learnt a lot about how article selection for offline wikipedia is done.
  4. Shot myself in the foot multiple times (over use of empty(), incomplete commit summaries and a tab/spaces issue). I’m rusty and it is showing. Shouldn’t be a problem anymore, though – because of you-know-what.
  5. Built myself a local copy of mediawiki docs. No more annoying huge-images.
  6. Got myself the solarized theme. And increased font size to normal levels (13pt). My eyes thank me.
  7. Fixed timestamp related bugs in SelectionSifter.

Devlog December 10, 2011

  1. Looks like I’m going to be rewriting SelectionSifter’s UI a lot. No more QuickTemplates, as I found TablePager. Mediawik definitely needs to fix Bug #1 soon.
  2. Added proper i18n for SelectionSifter.
  3. Redid FilterRatings page completely.
  4. Went through PhoneGap documentation, checked out state of its Photo handling API. Looks good.
  5. Checked out patch to official Wikipedia app that adds upload functionality. Needs lot more polish.

DevLog December 7, 2011

  1. Redid Blog template. Recategorized all blog posts
  2. Finally enabled mobile banking on ICICI. Still need 8 days for Internet activation. Sigh.
  3. Ordered HyperJuice 60W battery + Airline Adapter
  4. Read through the WPBannerMeta template (and its subtemplates), figured out where to get the assessment markers in. Poof that was hard. These templates are perfect examples of ‘hacks’ – initially ‘designed’ for templating text, they’ve grown into homebrew programming languages that are worse than PHP. True evolution :)
  5. Read through, replied to very detailed questions about MW development from the KovaiGeeks folks. Both me and Sumanah were amazed at the enthusiasm shown. Should be one fun workshop!
  6. Fixed arbit bugs in in SelectionSifter, moved away from anything in QueryPage
  7. Started using SparrowApp and WunderList. Amazing they are.

Story of Average Indian ‘Techie’

Note: I am just reformatting + putting this in one piece rather than have it split across multiple comments on reddit. Am not the original author

Beautifully laid out by someone in the under-trafficked r/india:

my state has 300+ engineering colleges and my parents decided that i am going to be an engineer. so, i wrote an exam and got into one institute (luckily i can make it into a good institute where there are campus interviews.. the luck being i might have gotten 2 multiple choice questions correct than an unfortunate guy who will be discarded by our society)

Here i am in college. Nothing big here, professors are very friendly and if you are good to them(i always made it a point never to refuse whatever they ask ,,, please do not think i am homo), they will give marks, or you can later request them and because your future is at stake they will give you some marks.

when the admin password for our oracle installation did not worked, my professor in DB class wondered if we could have tried it with some more s (**) instead of 6 *s(as in **). This is one of the A+ rated institute in the country and the professor is a phD in CS. No, the password is not made of * characters.

In the examination we are supposed to write the code (yes, the code) on paper and submit it. Most of the guys did scored more than 80% by writing an universal program (includes, defines, main (), some function names related to the problem, the usual i j k and l, loops and finally adding fflush () and return at the end ) nobody can dare try to understand what is in between.

We had this big campus placement drive, when we got the usual aptitude (if ram worked for 4 days and shyam for 5 days, in how many days the work is completed) and tricky questions ( can you write a one line c code to find if a number is even) and the most complex of them (string reversal, tree traversal) questions, leaked one day before from the previous college the company went to recruit. we all prepared well and some got lucky. most of the times, even if fail to answer any of the questions, but smile pleasantly, you got in. sometimes, even the smiling part it not necessary, you just have to act how stupid they are to ask such questions to you..

when we joined this big company, we got trained in how to use the PC and every programming language and every operating system and every technology, each one not taking more than a couple of days. We worked hard and successfully completed the training. the test is on a PC, but the question bank is known (airtel friends ad).

Now, we are put into a project, we are given a contact number of the manager and office location and were told to meet him there. we, group of 3 people (we are friends by now) try to locate the manager hard for 2 days and finally get hold of him. we explain him that we are sent by Ms.knowallthingsbutactuallydumb to work in (t)his project.

He jovially introduces us to the technical lead. TL says he will give work after some time, but meanwhile asks us to 1) collect all the birthdays,contact number, mail id, designation details of all the team members, make an excel sheet and mail it to everyone. 2)sit with a senior engineer who is doing testing (pressing keys and entering some form data in a webpage and checking the output) and we enter the data all day and give him the screenshots at the end of day.3)attend team meetings (why we are the only ones to get there first and wait for everyone for 30 mts) make MoM and email it to every one and track the action items (yes, now we know these terms.. you cant put us in noob mode)

Then this goes on for 4 months, still no real IT work. we are wondering why we are not given any work (this is the 8th month we are in the company, drawing salary). we try to get some real IT work from the seniors, but they always seem not to notice us and not taking us serious. But, being new to the environment and in doubt of our capabilities, we let this go. we make merry of the free time, enjoying the cafeterias, making (cute) friends from other teams, roaming around the big campus and then one day, the manager calls us all in. this is not the typical meetings we are experienced with, because the manager talked to us into the meeting and scheduled it as high priority calendar invite, only to us, the freshers.

Wondering if any of us did anything wrong (it is a relief to find ourselves not doing anything at all, still) we walked into the meeting. The manager is very blunt. Higher-ups (client and Delivery manager) is questioning about the productivity and inquiring what the freshers are doing in the project. Manager tells us that we need to show output and contribute to the project work. One member replies him that we never got any work assigned. Manager becomes more serious. he questions our lack of initiative and tells us that we should deliver results. He calls the team leader into the meeting and tells him that we should be working on any pending issues and asks him to track our progress.

We are scared, confused, and do not know what to do. then the TL takes us to his desk and on a white paper strip (of size A8) draws a diagram (Euclid would know better shapes) explaining to us the product, process, standards, and every other thing (you should never touch the code, you should never email anyone without it being shown to me first …) and gives all of us guest permissions to a directory/machine.

We are all happy now, we got into real IT work somehow (may be the manager is angry, but we can impress him with our work later and change it). we also heard, if we perform better we will get sent to the USA and can stay there for 3-6 months. this really motivated us. we immediately logged into the machines, started reading the code.

We could understand nothing. The code is nothing like we wrote or tried to write during our college days, it is not even like whatever is taught in training. everyone of us scared now, we the freshers. now we talked among ourselves, first to know if other guy had any answers and then to know if they dont. to approach the seniors, who always seem to be busy, either at their workstation or at the printers (i do not know why such a rush at the printer area) and to disturb them with our silly questions seemed like bad to us. so we turned to GOOG.

everyday we gone through each source file (php, html, JS, shell) and googled the meaning of everything we do not know and that is everything. the TL never bothered us after the first meeting. we are also making notes of the things and looking back, i think we could have made a wikipedia then. such was our effort, but at the end of a couple of months it did not made any sense to us, except that we now know how to write a hello world alert page in IE (sorry, it is in HTML, JS).

So, we completed 1 year in the company. our reviews are coming up but we have not done anything yet. again we are in a panicky stage. Meetings are going on, but we are just continuing taking the MoM and distributing it. Now and then, we are hearing some terms (forms, tables, elements, script, position, width, validation, anchor, cursor, layout …) we learnt from exploring GOOG. it made us feel good, but we thought we are not IN yet.

Somehow we finished our annual reviews, all of us got mid-rating and manager explained to us later that we do not get much hike as our ratings are average. One of us who stayed late hours in office because he came late everyday(he lived in some faraway place) got 1% more hike than us.

This went on for 3 more months, we using GOOG to learn some terms, and doing some job works to other team mates. like one day, i stayed overtime, took printouts of the client webpages (pages and different instances) and stacked them neatly, so the senior engineer can go through them (i do not know the need for this yet), and like this another day, when i have prepared this big excel sheet and entered Pass/Fail from another automated report and this excel sheet generated some more sheets with nice charts. Another friend of mine tracked the billing hours similarly..

Then, one day the Manager again called us. Asked us what we are doing, and it is good to see that he is very satisfied with our work. The TL also gave a good feedback and told the manager that we worked independently and always helped the team. The manager nodded, and told the TL to send our resumes to client and put us into billing. Again, we are worried. Tension is on our faces, what to put into the resume, how to answer any questions from the client…

TL asked us for our resumes immediately, we prepared them and sent it to him, but prayed to god, there be no client interview. TL called us in and told us that he modified our resumes (what a relief, but wait) and added Web technologies (meanwhile clients also know our names as we are in CC list and in some group mail lists) experience and Enterprise Apps (Oh, its time for GOOG again, for us) so that it will seem relevant to the client. We wondered what will happen, if the client asked us questions on those and we do not get selected. TL told us not to worry, he will take care of it…(this time we are courageous enough to ask the TL, because we do not want to bow our heads before a white guy)

Then, after 5 weeks (client manager is busy or on vacation hence the delay) it is informed to us that we got selected into the team. As we are already there for the last 1.5 years and working in the project and familiar with web and enterprise technologies, the client had no objection for us to be in the project. we felt so happy, manager and TL both congratulated us, and it felt like a new job joining day for us..

Now, everything is set. forget the education, trainings, friends, politics in office,, Now all that is left on the highway is to cruise at whatever speed we allow ourselves. We are all so happy, and one of the friends is already planning to move to a better paying company after 1 more year of working in Web and enterprise technologies. I am already thinking about when to ask for onsite opportunity, whether i should wait for couple of months or make my wish known to the manager right now itself?…

So, the D Day has arrived. it is the day(remember, the speech in Independence day movie).Hence forth, i will no longer be known as just me, but as a techie (forward 1 year i can add Sr. before that).

My assignment and the Project?? Usability Project.. testing the clients web platform for its usability and generating metrics and reports in rich media formats.(now, you know why the rush at the printer)

– anonymous

Sounds very true, from second hand experience of people who are working there. Anyone with first-hand experience to corroborate this?

DevLog: December 5, 2011

Am migrating to per-day posts now. More granularity + I can fit more info in. The ultimate aim of the DevLog is to keep me on my toes and make sure I don’t relapse into no-productivity hellholes. Daily DevLogs should do a better job at them.

  • Spent some more time trying to get Kiwix to compile under OS X. Macports issue fixed, but more issues persist. It’s one damn complicated build system :| Not to mention that OS X sucks balls. Still no luck.
  • Moved my extension from GPoC to SelectionSifter, as picked by Sumanah (and remembered by Siebrand)
  • Added more content to my ‘How to become a mediawiki hacker‘ page. Outline in shape, needs content. (This is the page I am rewriting)
  • Cleaned up my Mediawiki.org userpage. Added stuff I’m working on there. Archived lots of older pages.
  • Published livetrains to GitHub. As expected Kishore cleaned it up a bit. Have figured out how to do the time modelling (Thanks lifeeth!). Should finish up when I get time

Need.To.Be.More.Productive!