Pipeline http://juliuso.posterous.com A log of things I've worked on, and stuff to remember. posterous.com Mon, 20 Feb 2012 00:34:00 -0800 How to give access to your site arriving from other sites in Apache http://juliuso.posterous.com/how-to-give-access-to-your-site-arriving-from http://juliuso.posterous.com/how-to-give-access-to-your-site-arriving-from
Say I had a site that was external from a client's site. And the site could only be accessible if the user first went to the client's site, logged in, and then clicked a link to reach the external site. Can it be done?

The first thought might be to use straight ACLs to restrict and allow certain IP addresses access to the site. The problem with this is that it evaluates the end-user's IP address, and doesn't prove that the request is coming from the link provided on the client's website. In other words, the external site can only be accessed to users who are logged into the client's website, with access denied to those who directly try to access it, such as the public.

The solution is to make Apache evaluate the HTTP Referer (misspelled intentionally), or the SOURCE of where the link was clicked on. 

Just drop this is in your .htaccess file of the site's root, or subdirectory, and you're ready to go.

SetEnvIf Referer EXAMPLE.ORG example
Order Deny,Allow
Deny from all
Allow from env=example

Substitute EXAMPLE.ORG for the originating site. "example" is the variable name.

No need for fancy cross-site authentication. It's simple, and does the job.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Tue, 14 Feb 2012 01:04:00 -0800 Euler 22 solved http://juliuso.posterous.com/euler-22-solved http://juliuso.posterous.com/euler-22-solved

Solution

Lots of time was wasted solving this one, because I continue to be in trenches of solving the problem with one large procedure, which isn't the way to do it.

My approach was to:

  1. Get the text file copied into a list.
  2. Convert all the letters to numbers and store them in one large list.
  3. Create another list that will map all 26 capitalized letters to numbers from 1-26.

Then I hit a roadblock (or a dumpster from The Big Lebowski) because I was clueless as to how I could slice a list with a variable length, while also noting the row number (or the index) for each name, to make the calculations.

They key was to make a function that processed each name in the list, record all the necessary values, and perform the math. A for loop was all that was needed to pass each name into the function, and store the sum in a new list outside the function, which creates a list of subtotals, from which a grand total could be made.

So lessons learned from this one. Don't try to wing it with one large procedure. Break it up into smaller steps. Be modular, use functions!

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Wed, 08 Feb 2012 14:42:34 -0800 Euler 21 solved http://juliuso.posterous.com/euler-21-solved http://juliuso.posterous.com/euler-21-solved

Solution

Another quick win, I love it!

Nothing fancy here. The first version of the problem was returning two additional values above 10,000 that were beyond the problem's scope, so the solution had to be refined. But I did sum up all the numbers below 10,000, enter it into Euler, and it was correct. I wanted to be sure I was taking the right approach.

On to 22.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Wed, 08 Feb 2012 14:29:00 -0800 Keep iterator accumulators OUTSIDE loops! http://juliuso.posterous.com/keeping-increment-and-decrement-variables-out http://juliuso.posterous.com/keeping-increment-and-decrement-variables-out

I don't use while loops often. But when I do, I forget that any index iterator has to be OUTSIDE the while loop. Otherwise, the iterator never properly increments/decrements the index.

Add a print statement for the iterator in the loop, and run the program. If you see the same value repeating itself (most likely the iterator's initial value), then you know it's not working.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Wed, 08 Feb 2012 12:41:28 -0800 Run and edit a Python script in vim http://juliuso.posterous.com/run-and-edit-a-python-script-in-vim http://juliuso.posterous.com/run-and-edit-a-python-script-in-vim

I'm an ok vim user, but I'm in TextWrangler and TextMate most of the time. But if you're editing a Python script in vim, and want to quickly run it, do this:

:!python %

I'm sure it could be shortened so there's less to type. Or press : then Up Arrow to recall the last command.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Mon, 06 Feb 2012 00:46:00 -0800 Perl Fundamentals for Non-Perl Programmers http://juliuso.posterous.com/perl-fundamentals-for-non-perl-programmers http://juliuso.posterous.com/perl-fundamentals-for-non-perl-programmers

Curiosity lead me astray this weekend as I got curious about Perl. I don't know much about Perl, except that it shares a history with UNIX as going hand-in-hand for all your scripting, and automation needs. Don't feel like reading the wiki article on Perl, but I'll guess that Perl could do anything you wanted that traditional shell scripting couldn't do.

This is a very good primer from Sam Hughes that takes you through Perl's fundamentals. Takes about 2.5 hours, but was longer for me since I took notes and tried some of the examples through Perl's interactive debugger. This is well-suited for those not from a Perl background that may encounter it in the future, whether it be to fix a broken script on an old production system, or to migrate the programs to another language such as Python.

From an outsider's perspective, Perl is challenging because there's more than one way of doing something, and one must understand the module system, and the behaviors of $scalars, @arrays, and %hashes during manipulation.

If you want to try Perl without writing scripts, use this command to start the interactive debugger:

perl -d e 1

It was a good crash course, and I won't feel so confused or uncomfortable if I ever have to be in that situation. But I'm fortunate to be in the Python camp.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Mon, 06 Feb 2012 00:07:00 -0800 Euler 20 solved http://juliuso.posterous.com/euler-20-solved http://juliuso.posterous.com/euler-20-solved

Solution

Sometimes, a quick win's necesary to keep the fire burning, stay focused, and get in the zone. Euler 20 was one of them, done in less than hour, rather than days, or MONTHS!

The code speaks for itself, but wanted to note:

  1. I second-guessed using math.factorial() because it was ridiculous to add overhead to the code, when the factorial could be calculated with a simple for loop. No modules necessary.
  2. It's not possible to iterate over a long. Or in Pythonese, is non-subscriptable. Converting the calculation to a str() did the job.
  3. When I started solving these problems, I took the liberty of storing every new calculation, or transformation into a new variable. Again, not a good practice unless it's necessary to see a progression of how the transformation occurred. The factorial was then converted from a long to string. As we iterated over the string's indices, each element was converted to an int to store the sum of the index's elements.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Tue, 31 Jan 2012 18:38:00 -0800 Figuring out why I kept getting IndentationErrors in Python http://juliuso.posterous.com/figuring-out-why-i-kept-getting-indentationer http://juliuso.posterous.com/figuring-out-why-i-kept-getting-indentationer

I forgot to mention last night that I started getting IndentationErrors in Python while working on Euler 19. It was frustrating because all the code was nested properly, and I looked at the lines directly above and below where the interpreter indicated the error.

And then it hit me after reading this at Stack Overflow. I recently changed TextWrangler's indention to use 4 spaces instead of tabs, and have used tabs since starting to learn Python 2 years ago. But after reviewing PEP 8, I felt it was right to follow the predominant standard.

I did a quick Find/Replace, finding \t's, replacing them with [space][space][space][space], and all was well in the world again.

I'll remember to keep this in mind when I go back to previously solved Euler problems to change/optimize them. Spaces and tabs don't mix.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Tue, 31 Jan 2012 02:45:00 -0800 Euler 19 solved http://juliuso.posterous.com/euler-19-solved http://juliuso.posterous.com/euler-19-solved

Solution

I'm happy to put this one to bed. Before this, I solved problem 18 almost THREE months ago. Geeze, it's unbelievable how long it took to figure it out! Glad I'm still sticking to my promise of solving one Project Euler problem per month. Better start 20 immediately.

It isn't elegant, but gets the job done. Once I know more, I hope to return to these and optimize them further. A lot of data was duplicated in the process, and hope it can be reduced in future iterations. Marc first brought this to my attention when I was using a for loop to store values in an array, and using another for loop to apply conditional logic tests. His suggestion was simple, why not evaluate the range of values inline, and skip building arrays altogether? Brilliant, but in this case, not so brilliant.

Here's my approach to solving the problem:

  1. Take each year, and determine if it's a leap year. Based on the problem's definition of leap year, a function was built that returned True if it was a leap year, False if it wasn't. In this case, False would be equivalent to a standard, non-leap year. And while the problem specifically asks for the number of occurrences where the 1st was on a Sunday betwen 1901 and 2000, I calculated 1900 as well, because they give 1-Jan-1900 as the starting point in the week, a Monday. That means that the last day of the week is Sunday, the sevneth day.
  2. Each month is defined as a variable, with it's number of days. The only month affected in leap years is February, 29 in a leap year, and 28 in a standard year.
  3. Create an array for all the calendar days in a leap year and standard year.
  4. Based on the conditions of 1, a dictionary is created that lists all the days of the year based on 3. Each dictionary key is a year, with its value as an array of all the calendar days in that year. The dictionary contains a key/value pair from 1900 to 2000, inclusive.
  5. Only the values from the dictionary are taken to create a single array that contains all calendar days from 1900 to 2000.
  6. Remember that since 1-Jan-1900 starts on a Monday, Sunday will always be the 7th day of the week, and continues through the series. Like a cookie cutter, we'll go ahead and use the slice operator to chop-up the array from 5 using x[6::7], return only the seventh number in that segment, and store it in another array.
  7. The array created from 6 is tested. If any element equals 1, that means that the 1st of that specific month started on Sunday. We keep a counter going and increment by 1, as needed.
  8. Becuase the counter from 7 includes all occurences of Sundays from 1900-2000, a separate function is created to count the occurrences of 1st on Sundays in 1900 only since the problem asks to find the occurences from 1901-2000. We keep a counter for this as well.
  9. The final answer is the total number of occurrences, minus the occurrences in 1900.

Date and time arithmetic is interesting, and hope to get better with it over time.

Made some progress on Codecademy today. The only problem is that results seem to very from browser to browser. So a lesson that fails to complete in Safari may work in Firefox or Chrome. Since JavaScript implementations are different from browser to browser, your mileage may vary. So if you think what you typed is correct, but still gives an error, try another browser and it should work.

 

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Sat, 28 Jan 2012 12:21:00 -0800 Codecademy tutorials http://juliuso.posterous.com/codecademy-tutorials http://juliuso.posterous.com/codecademy-tutorials

I've been having fun going through the tutorials on Codecademy because I was intrigued by what kind of teaching approach they would use. Short lessons of reading text, and requiring the student to type code into a window to prove their understanding seems to be a great approach to keep a student's attention and momentum going. The speed and pace of how quickly I've been going through these JavaScript tutorials has been satisfying, and goes to show that once one knows a programming language, learning other languages is easier.

For the novice programmer, an understanding of the fundamentals is the key to understanding all the other languages out there: creating/manipulating variables, iterating through arrays, using functions to modularize code, knowing there may be a library of pre-written code that you can use to solve your problem, etc. There's no need to feel anxiety or frustration. Just start learning one language, and understand it as best you can. 

From the SICP perspective, a langauge can be understood by knowing its primitives, the means of combination, and the means of abstraction.

Once you know this, you know the rules of the game.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Tue, 24 Jan 2012 19:01:07 -0800 Dreaming of Model-View-Controller http://juliuso.posterous.com/dreaming-of-model-view-controller http://juliuso.posterous.com/dreaming-of-model-view-controller

A few days ago, I had a dream about doing something in Ruby on Rails (RoR). From what I remember, I was working somewhere, and was compelled to write an app to solve some problem. Once I started digging into it, it was over, and the dream ended.

It's strange to dream of stuff like this, but does make some sense because it occurred 1 week ago today, on the day I wrote about the stuff I was working on, and that doing web apps, and understanding the MVC development model wasn't a priority of mine.

So while I de-prioritized learning this, my subconscious was telling me that I should. So from it, I took what I already knew and started digging into Django's documentation, reading the beta second edition of the Django book here, and following the examples. And what a challenge it is working with beta documentation, and figuring stuff out on your own!

Why didn't I follow what the dream said, and dig into RoR? Well, I felt I needed to start on familiar footing, and with already knowing Python, Django seemed like a good start, aside from the many stalled attempts last year to figure it out. I'm not knocking RoR are in any way. It's just that I'll get around to it, in time.

It's been a very productive week, and things have really started to click-in. Sometimes, I have trouble with the fundamentals of learning new mental models. In this case, I think it's especially difficult, or almost impossible to understand something like this without having some prior programming experience. I'm just glad Project Euler gave me an outlet to solve actual problems, and get a better feel for a langauge. The end result is a greater understanding of these higher-level concepts, without being overwhelmed by the code, it's placement, or its syntax.

This is a big milestone because MVC is the basis for these web frameworks, and also is the model for writing Cocoa/iOS applications. While my journeys, and experiments in parallel learning continue, I hope to get through enough of it so that I can start creating, and deploying stuff.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Wed, 18 Jan 2012 23:13:00 -0800 Exercise and Insomnia? http://juliuso.posterous.com/exercise-and-insomnia http://juliuso.posterous.com/exercise-and-insomnia

It's true, and it happens. If you've started a serious exercise routine, and have sleep problems, look at this:

http://paleohacks.com/questions/7494/exercise-induced-insomnia

I had trouble sleeping during the first round of P90X because I was on a low carb diet, and wasn't eating enough. It started happening again in the second round, and is one of the worst feelings to have because the body is never comfortable lying down. I tend to believe this was one of the reasons why I didn't get the results I was expecting, because I wasn't getting good quality rest.

I tried exercising today, but had to stop because I was still incredibly sore from the Back & Chest, and Plyometrics exercise earlier this week. Even taking the stairs was tough.

So make sure you eat a lot, and get some carbs and fluids in you before sleeping. I may resort to using 5-HTP + Melatonin if things don't improve.

It makes sense that you have to eat and drink a lot in these programs because of the calorie burn that depletes you of everything, and having to take in more nutrients to get everything you lost, and hope to have some energy in reserve.

So, don't feel bad. Eat more!

Add the occasional exercise tip to the blog too. It feels right.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Tue, 17 Jan 2012 14:22:00 -0800 Current Projects http://juliuso.posterous.com/current-projects http://juliuso.posterous.com/current-projects

Here are the long-term coding projects in my pipeline of stuff:

Python

I've been using Python for about 2 years, and would say that I'm ok with it. The jobs I had didn't have much opportunities for programming in the language, so I had to find ways outside of work to work on non-trivial code. Project Euler has been great, especially for those with an interest in Math. The goal is to solve 1 Project Euler problem monthly. Anything more is bonus.

here was also a repeatedly stalled effort to write web apps in Django, but the lack of motivation to design the backend and front-end made it a chore. I'll get around to it someday, just not the right time for it. Add Ruby on Rails to the mix as well.

Objective-C

My knowledge in Obj-C is limited at present, but decided to dig in by learning the fundamentals. I read Kernighan & Ritchie C over the holidays, and now reading Cocoa and Objective-C: Up and Running by Scott Stevenson. Guess I'm at the stage where I just don't get it yet, and hope that once this hurdle is overcome, I can dig into writing apps, starting off with simple ridiculous ones. Afterwards, the books from the Big Nerd Ranch seem to be the logical next step, digging into iOS's APIs.

Lisp

I've played around with Scheme a little bit, and have watched the Lisp class taught by Abelson & Sussman. It really piques my curiosity, with its application in AI research, and that pg used it to make Viaweb.

But I have to wonder if Lispers are also big fans of HP RPN calculators.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O
Mon, 16 Jan 2012 00:53:37 -0800 Introduction http://juliuso.posterous.com/introduction http://juliuso.posterous.com/introduction

I stopped blogging a few years ago because I felt the writings had no sense of direction, was unfocused, and the process unnatural. The best I could do was write a weekly essay for about six months straight on a variety of subjects before running on empty.

Now that I've had time to think about what to write about, I thought it would be nice to keep it within the realms of Science, Technology, a touch of Philosophy, and my journeys in programming. That's about it.

The intended audience of this blog is mostly for me. It's to keep a log of stuff I'm doing, to write aloud, talk about the ebbs and flows of problem solving, and working on projects. I've benefitted tremendously over the years from Googling so many things, usually landing on someone's blog that had the answers I was looking for. I want to give back, and hope that in time, it will help someone in their research across the Interwebs.

There's also an element of keeping productivty up, staying accountable, and self-managing oneself. I hope to establish some correlation between the frequency/volume of postings, to a matching level of productivity.

Let's begin.

Permalink | Leave a comment  »

]]>
http://files.posterous.com/user_profile_pics/1750135/JO-Thumb2.png http://posterous.com/users/ehbVO3uvgaILE Julius O juliuso Julius O