Python, Subprocess and Multiple Arguments

Calling external programs from within a python script is a pretty common task. However, most of the times when I want to pass some arguments to the external program, I usually get stuck. The official Python documentation for the subprocess module is a bit lacking, in my humble opinion, in its treatment of argument passing.

This is not to say that all the information isn't available in the official documentation. In fact, it is. It's just not where I always look for it. The first thing I look for are examples. I simply do not have time to read the entire documentation page for something which should not be too difficult.

The basic idea is this:

1. Either provide a string of the entire command and argument, but use Shell=True
2. Provide a string for only the command
3. Provide a list of strings, where the first item is the command and the other items are the arguments

It's the third option that keeps on giving me trouble, so here is a simple working example for reference:

 1 #!/bin/env python
 2 import subprocess
 3 
 4 program_name = "ls"
 5 arguments = ["-l", "-a"]
 6 
 7 command = [program_name]
 8 command.extend(arguments)
 9 
10 output = subprocess.Popen(command, stdout=subprocess.PIPE).communicate()[0]
11 print output

Passing Arguments with Quotes
If you want to pass arguments that are passed with quotes in the shell, then just pass them as a single list item, without the quotes.

Links

1. James Gardner wrote the most comprehensive introduction to the python subprocess module I've encountered.
2. The official documentation.
   

Further Reading

1. Bash: Passing Arguments with Quotes
2. Why Not to Write Shell Scripts
   

Learning Japanese - Resources

Selected Articles

1. Memorization - how to memorize kanji, words and grammar.
   

Guides & Blogs

1. All Japanese All the Time - this website has tons of articles about how to learn Japanese by having fun and immersing yourself in the language 24/7. The author claims to have learned Japanese in 18 months while studying for a computer science degree in an American college.
2. Nihongo Pera Pera - includes many articles on how to study Japanese by yourself. Includes many tips, especially on how to use the computer to its fullest potential.
   

Grammer

1. Tae Kim's guide to Japanese grammar contains an excellent guide with plenty of examples.
   

Computer Software

1. Smart.FM - for a free registration, you get access to tons of free online material. I'm currently studying a list of 2000 core Japanese sentences. The online learning system, called iKnow, has example sentences and recordings by native speakers. See my article on how to use it to learn vocabulary.
   
2. Reviewing the Kanji - this website is a great companion to Remembering the Kanji. You can select how many lessons you've already learned, and then quiz yourself. The real bonus is that for each character, you have access to user created stories. These stories can be voted on, and this means that you can easily find some really great stories!
3. Anki or Mnemosyne - both of these are completely free space repetition software (SRS). Anki is very polished and overflowing with options, whereas Mnemosyne is much simpler. Both offer a wide collection of preexisting cards. Anki also has version for some portable devices and a free online account which can synchronize between several computers.
4. Rikaichan - an excellent firefox plugin that gives you information on kanji and words whenever you hover your mouse over them.
   

Stories in Japanese

1. Free stories can be found at this website. The stories are short and simple, and there are plenty of them to choose from.
2. White Rabbit Press offers a series of booklets graded according to the JLPT vocabulary. The kanji characters have furigana, and there's a bundled CD with a native speaker reading each story. Each set includes 5 booklets.
   

Learning Japanese - Memorization

I've been studying the Japanese language for nearly a year now. I didn't learn as much Japanese as I'd liked to have, but I did learn a great deal about learning.

In a previous article, I listed my early attempt at learning how to memorize. I've learned a lot since then, and I'd like to share it with you. It boils down to three main tips:

1. Use associative memory
2. Learn in context
3. Use SRS software

Associative Memory

The Japanese writing system consists of several "alphabets" used simultaneously. The most famous is the kanji writing system, which is made up of thousands of characters. In order to read a newspaper, you need to be familiar with about 2000 different characters. That's quite a lot of letters to remember.

My initial approach was to learn by repetition. I'd write each character dozens of times and then quiz myself. This method not only takes a lot of work, but it's very easy to forget characters after some time without review. I can say that after about 200 characters, it was getting too time intensive to continue.

I recently started using Remembering the Kanji by James Heisig. The book promotes using our imagination to better memorize.

Each character has a keyword attached to it, and the complex characters are made up of combinations of simpler characters. Therefore, in order to remember the complex "letters", you make up some story that connects the keywords of the simpler letters and the keyword of the complex letter.

The keyword of the simpler characters are remembered using the keywords of even simpler characters. This all goes down until the axioms, sorry, the primitive elements.

It is now very easy to remember how to write a new character. For example, to remember how to write "bull's eye" I just image myself throwing a white ladle at a target and hitting the bull's eye. To commit this to memory, I spend several minutes picturing myself standing all tense, with a large audience watching, the fate of the free world resting on my ability to hit the bull's eye with a shining, white ladle. Now, I know that to the letter with the keyword "bull's eye" is made up of the letters for white and lade.

The serious beginning in the story is in direct contrast to the silly image of throwing a ladle, which only helps burn the story into memory. I try to spend a couple of minutes really imagining the story. I also attempt to make the story as interesting and bizarre as possible, preferably involving as many senses as possible.

So far this technique has been working miracles for me. I intend to finish learning all of the basic 2000 characters in half a year, and I really believe its possible!


Free Resources:

1. The first few lessons of Remembering the Kanji are available for free.
2. There is a website where you can read the top rated stories people invented.
   

Further Reading:

1. Learning in context - to be written
2. Using SRS software
   

Configurability

A while back I stumbled upon the following vim command, which opens a file explorer view. Now, this is truly scary stuff. I was as shocked and disappointed as you probably are. Assuming you're me, that is, and you're most likely not me. You just might be that other person who reads this blog, in which case you probably have no idea what I'm whining about. Alas, to explain, we have to go a little bit back in history.

The best way to travel back in time is the "Once Upon A Time - Man" way. We'll use Peter, which Wikipedia tells us is the very embodiment of the Good Man, as our guide.

In ancient times Peter lived with his entire extended family in the same house. Maestro was the leader of this establishment, and everyone, from Jumbo to Pierrette, went to sleep at night under the same roof. It was one big happy family.

As time went on, and social orders changed, the atomic unit shrank to include only the immediate family. Peter had to leave his parent's house and make his own way in life. After finding a lovely life (people who are Good have beautiful wives), he would get his own place and raise his children there. Until they got kicked out because they were lousy bums who had to get a job.

Nowadays, Peter and his present day wife would probably get divorced, so the family unit is now as small as it gets, unless the universe gets very creative.

Okay, we finished our time travel, and you probably have no idea why a little file explorer freaked me out. I guess we'll just have to make the same journey again until you get it.

A couple of thousand years ago, after the petty city states finished their squabbling, it was pretty nifty to be a king. As a king you were a member of a very small club. There just weren't many kings back then. Every once in a while someone would go around conquering half the world, limiting the number of nations and thus the number of kings.

This continued until just after the world wars and the rise of nationalism, which resulted in everyone wanting their own state. Wolfram Alpha says there are 203 sovereign states right now. Unfortunately, Wolfram Alpha can't answer the simple query: "Number of countries as function of time", but neither can google. You can imagine that over the past centuries this number is an increasing function.

Now, not only does the number of states increase, their power decreases as well. When once a totalitarian state was a pretty simple matter, the internet has made it much more difficult. Not only because Iranians protesters can twit, but also because Hezbollah backed Palestinian terrorists can coordinate attacks and learn how to make bombs. The state is no longer omnipotent.

One more diversion before I get to the point, which should be quite anticlimactic. When Henry Ford made the Model T, all the cars were black. Now they come in a wide range of colors. In addition, everyone wants his very own custom T shirt. I'll leave Apple out of it for the moment, because I'm going to get back to them soon. Everyone besides Apple wants you to have a unique user experience, complete with a customizable theme.

The Point. When I saw that Vim has a file explorer, I was terrified that it would go the way of the emacs. I hate it when a software not only allows, but actually expects you to customize everything in it. This is why I was afraid that Vim was starting down the road that would make it a good OS (albeit lacking a good editor). Emacs tries to do everything, and it tries to let you do everything. That just sucks, no matter what estimmed blogger Steve Yegge says.

For example, I mapped my Win key to the escape key, so that I can quickly switch between normal and insert mode. The annoying thing is that this only works on my machines. When I use a different machine, things suck. This is why I hate Configurability. I prefer uniformity. And I much prefer stuff That Just Works. This is the Apple part I promised two paragraphs back.

I could go on for much longer on this issue, and I probably should. Alas, I've written too much stuff already, without even explaining how everything relates, and the wife deserved some attention. More later.

p.s.
The file explorer in Vim is ok. It's just a file open dialog box. So Vim is still good.

One Year Summary

A bit over a year ago I started this blog. I didn't know much about blogs and twitter and such at the time. After a year of blogging, I'm afraid I still don't. I think my main pain is that I still write these posts in the damn web UI. I'd much rather write in vim, but I haven't found the time yet to figure out the best way to do it.

During the past year I made many public promises to myself, on this blog and the others. I haven't really lived up to all of them. I have a consistent problem of trying to do too many things. Working full time and studying Japanese have taken up nearly all of my time.

While I didn't get down to writing a game in Python like I wanted to, I did manage to learn the language. Luckily, it was a perfect fit for many of the tasks I had to do at work, and it was a joy to learn it. I have my gripes with the language, but since I work in C++ most of the time, its a very welcome change. Needless to say, Python code is much more readable. C++0x has the auto keyword which will help, but a static language will never be able to match a dynamic one in this criteria.

I intend to continue studying Japanese, though maybe not in a university framework. I have to figure out if I want to start studying for a Master's in Physics or not. None of the courses I can take will be easy in conjunction with a full time job. I may end up preferring not to take any official obligation and spend time studying the things I like by myself.

We'll see how the next year will come along...